{"id":5098,"date":"2024-07-25T00:44:50","date_gmt":"2024-07-25T04:44:50","guid":{"rendered":"https:\/\/ikriv.com\/blog\/?p=5098"},"modified":"2024-10-04T10:51:49","modified_gmt":"2024-10-04T14:51:49","slug":"python-serializing-json-into-dataclasses-with-validation","status":"publish","type":"post","link":"https:\/\/ikriv.com\/blog\/?p=5098","title":{"rendered":"Python: serializing JSON into dataclasses with validation"},"content":{"rendered":"<p><b>TL;DR<\/b> use <a href=\"https:\/\/pypi.org\/project\/marshmallow-dataclass\/\">marshmallow-dataclass<\/a>.<\/p>\n<p>Serializing JSON into dataclasses with validation proved to be unexpectedly difficult. By &#8220;validation&#8221; here I mean type checking (this must be a valid integer), range checking (this integer must be non-negative), length checking (this string must be at least 5 characters long), and things like.<\/p>\n<ul>\n<li>&#x1f44e; <a href=\"https:\/\/marshmallow.readthedocs.io\/en\/stable\/\">Marshmallow<\/a> is very rich in features, but it can&#8217;t do dataclasses, it produces dictionaries.<\/li>\n<li>&#x1f44e; <a href=\"https:\/\/pypi.org\/project\/dataclasses-json\/\">Dataclasses-json<\/a> is based on Marshmallow, can do dataclasses, but I could not find how to do validation, the docs do not mention it.<\/li>\n<li>&#x1f44e; <a href=\"https:\/\/docs.pydantic.dev\/latest\/\">Pydantic<\/a> offers validation, but seems to be not compatible with type checkers, see below.<\/li>\n<li>&#x1f44d; <a href=\"&quot;https:\/\/pypi.org\/project\/marshmallow-dataclass\/\">Marshmallow-dataclass<\/a> looks like the right thing.<\/li>\n<\/ul>\n<h2>Marshmallow-dataclass<\/h2>\n<p>Marshmallow-dataclass allows access to underlying Marshmallow validation as follows:<\/p>\n<pre class=\"brush: python; title: ; notranslate\" title=\"\">\r\nfrom dataclasses import dataclass, field\r\nfrom marshmallow import validate\r\n\r\n@dataclass\r\nclass Person:\r\n   age: int = field(metadata={&quot;validate&quot;: validate.Range(0,150)})\r\n<\/pre>\n<p>This is a little verbose, but it gives access to the entire spectrum of Marshmallow validators. The only caveat is that a validated field is considered to have a default value, so all fields after it must have a default value as well. This can be fixed by making the default value just <code>field()<\/code>:<\/p>\n<pre class=\"brush: python; title: ; notranslate\" title=\"\">\r\nfrom dataclasses import dataclass, field\r\nfrom marshmallow import validate\r\n\r\n@dataclass\r\nclass Person:\r\n   age: int = field(metadata={&quot;validate&quot;: validate.Range(0,150)})\r\n   name: str = field()\r\n<\/pre>\n<p>Marshmallow-dataclass is very hands-off, the only place where you actually need it is to get a schema from the dataclass. Here&#8217;s a full examle:<\/p>\n<pre class=\"brush: python; title: ; notranslate\" title=\"\">\r\nfrom dataclasses import dataclass, field\r\nimport json\r\nfrom typing import Optional\r\nfrom marshmallow import EXCLUDE, validate, ValidationError\r\nimport marshmallow_dataclass\r\n\r\n@dataclass\r\nclass Address:\r\n    street: str\r\n    city: str\r\n    zipcode: str\r\n\r\n@dataclass\r\nclass Person:\r\n    name: str\r\n    address: Address\r\n    age: int = field(metadata={&quot;validate&quot;: validate.Range(0, 150)})\r\n    profession: Optional&#x5B;str] = None\r\n\r\njson_str = &#039;&#039;&#039;\r\n{\r\n    &quot;unused&quot;: &quot;bla-bla&quot;,\r\n    &quot;name&quot;: &quot;John Doe&quot;,\r\n    &quot;age&quot;: 30,\r\n    &quot;address&quot;: {\r\n        &quot;street&quot;: &quot;123 Main St&quot;,\r\n        &quot;city&quot;: &quot;Springfield&quot;,\r\n        &quot;zipcode&quot;: &quot;12345&quot;\r\n    }\r\n}\r\n&#039;&#039;&#039;\r\n\r\ndata = json.loads(json_str)\r\nschema = marshmallow_dataclass.class_schema(Person)(unknown=EXCLUDE) # allow extra fields\r\ntry:\r\n    person = schema.load(data)\r\n    print(person)\r\nexcept ValidationError as e:\r\n    print(f&quot;VALIDATION ERROR: {e}&quot;)\r\n<\/pre>\n<h2>Pydantic<\/h2>\n<p>Pydantic offers special field types like <a href=\"https:\/\/docs.pydantic.dev\/latest\/concepts\/types\/\"><code>conint<\/code><\/a> to express validation:<\/p>\n<pre class=\"brush: python; title: ; notranslate\" title=\"\">\r\nfrom pydantic import dataclass\r\n\r\n@dataclass\r\nclass Foo:\r\n   positive_int: conint(ge=0)\r\n<\/pre>\n<p>This works great for validation, but type checkers like mypy are not happy about it, they complain that function calls are not allowed in type annotations. They also can&#8217;t figure out that <code>positive_int<\/code> is an int. Pydantic offers <a href=\"https:\/\/docs.pydantic.dev\/latest\/integrations\/mypy\/\">a plugin for mypy<\/a> that allegedly fixes it, but I haven&#8217;t tried it.<\/p>\n<h2>Conclusion<\/h2>\n<p>Serialization into dataclasses should not be a mine field, it should be part of the standard library. Fortunately, marshmallow-dataclass provides a reasonably good solution.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>TL;DR use marshmallow-dataclass. Serializing JSON into dataclasses with validation proved to be unexpectedly difficult. By &#8220;validation&#8221; here I mean type checking (this must be a valid integer), range checking (this <a href=\"https:\/\/ikriv.com\/blog\/?p=5098\" class=\"more-link\">[&hellip;]<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"Layout":"","footnotes":""},"categories":[4,26],"tags":[],"class_list":["entry","author-ikriv","post-5098","post","type-post","status-publish","format-standard","category-hack","category-python"],"_links":{"self":[{"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/5098","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=5098"}],"version-history":[{"count":10,"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/5098\/revisions"}],"predecessor-version":[{"id":5126,"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/5098\/revisions\/5126"}],"wp:attachment":[{"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=5098"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=5098"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ikriv.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=5098"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}