Marshmallow-dataclass: eliminating nulls in the resulting JSON

When serializing a dataclass, by default marshmallow-dataclass emits nulls for all fields that have the value of None. This is not necessariy most of the time and pollutes the resulting JSON.

There seems to be no easy way to prevent generating the nulls, but we can remove them after the fact by creating a special base schema with a post_dump hook.

Example:

from typing import Optional
from dataclasses import dataclass
from marshmallow import Schema, post_dump
from marshmallow_dataclass import class_schema

class RemoveNullsSchema(Schema):
    # remove None values from the generated dictionary, which eliminates nulls in the final JSON
    @post_dump(pass_many=False)
    def postdump(self, data, **kwargs):
        for key in list(data.keys()):
            if data[key] is None:
                del data[key]
        return data

@dataclass
class Point:
    x: float
    y: float
    name: Optional[str] = None
    
p = Point(2,3)
regular_schema = class_schema(Point)()
print(regular_schema.dumps(p))  # {"x": 2.0, "y": 3.0, "name": null}  

enhanced_schema = class_schema(Point, RemoveNullsSchema)()
print(enhanced_schema.dumps(p)) # {"x": 2.0, "y": 3.0}  

6 Comments


  1. list(data.keys()) seems like an necessary call. No?

    Reply

    1. Sorry: “necessary” -> “unnecessary”.

      Reply

  2. Maybe data.items() instead of data.keys()? Then None check can be done w/o lookup…

    Reply

    1. We are modifying the data inside the loop. So, data.items() wouldn’t work, you’d have to do list(data.items()), which means copying keys AND values (just references, but still). But indeed, if you copy the values, you won’t need to do look-ups. It’s basically trading memory for performance, one could go either way.

      Reply

      1. Got you.
        I lost touch with Python. You _can_ delete while iterating in Go.

        Reply

        1. In C# you can’t delete, it will raise an exception.
          In Python you can delete, but it will silently skip items.
          C# and Go at least got their act together, Python’s behavior is not ideal.

          Reply

Leave a Reply

Your email address will not be published. Required fields are marked *