Skip to content

Any None value make Delta non safely serializable #224

Closed

Description

@Annakan

Describe the bug
If a data structure contains a None value the serialize/deserialize roundtrip breaks

To Reproduce

Delta(Delta(DeepDiff({"a":None}, {"a":1})).dumps())

deepdiff.serialization.ForbiddenModule: Module 'builtins.type' is forbidden. You need to explicitly pass it by passing a safe_to_import parameter

Although there is builtins.None in SAFE_TO_IMPORT, pickle need to use builtin.type for reason I did not figure yet

This works :
Delta(Delta(DeepDiff({"a":None}, {"a":1})).dumps(), safe_to_import={'builtins.type'})
=>
<Delta: {'type_changes': {"root['a']": {'old_type': <class 'NoneType'>, 'new_type': <class 'int'>, 'new_v...}

But of course allowing builtins.type opens the insecure floodgates wide open, since any class can then be instanciated.

Expected behavior

Delta(Delta(DeepDiff({"a":None}, {"a":1}), safe_to_import={'builtins.type'}).dumps())
=>
<Delta: {'type_changes': {"root['a']": {'old_type': <class 'NoneType'>, 'new_type': <class 'int'>, 'new_v...}

OS, DeepDiff version and Python version (please complete the following information):

  • OS: [Manjaro]
  • Version [rolling]
  • Python: 3.9.1

Additional context
Deepdiff and delta are amazing ;)

Side note : the documentations at https://zepworks.com/deepdiff/5.0.0/delta.html#delta-safe-to-import-label seems incorrect
delta = Delta(t1, t2, safe_to_import={'mypackage.mymodule'}) implies that the safe_to_import parameter acts at Delta creation time (and it looks like DeepDiff syntax) and it is not.
It acts (and that's logical) at de-serialization time :
delta = Delta(delat_dumps, safe_to_import={'mypackage.mymodule'})
Maybe Delta init does a little too much magic, Delta.from_diff() and Delta.unserialize() or other explicit classmethod whould make use more "intuitive".

Activity

  1. seperman commented on Jan 14, 2021

    @seperman
    Member

    @Annakan Haha thanks. Happy to hear you like DeepDiff and Delta.
    Yep the safe_to_import is just for the import time. I will update the docs. And I agree that the init does too much magic. I just wanted the user interface as simple as possible which goes against being explicit. We can probably have some factory method exposed for people who want it explicit. What do you think? Since users have already started using the current api, I don't want to make any breaking changes to the api.

    Also here is a PR to fix the issue you brought up:
    #225

  2. Annakan commented on Jan 19, 2021

    @Annakan
    Author

    Thanks for your prompt answer.

    I think the ClassMethod/Factory method would be an improvement.
    And while you are at tit ... moving the code from the delta.add into a "proper" classical method like apply_to(object, mutate= ...) and having add calling this method would also make the interface more clear especially when mutating in place :).
    For instance having :

      my_delta.mutate = True
      mydelta + my_object

    looks a bit weird and takes two steps, and we need to be careful of + parameters order.
    Instead ```my-delta.apply-to(my_object, mutate=True) looks more natural in that case.

    And having a non commutative + is counter intuitive for many people given that all the base types that implement add are, unless I am mistaken.

    Also maybe mutate does not really belong in the Delta, it is more a use time decision rather than a creation time decision to me.
    For instance, In my use case I deepcopy once the source object and then apply a string of mutating Deltas rather than having each Delta deepcopy a big structure.

    Another idea for a Delta ClassMethod apply_all which would take and object, an iterable of deltas and mutable, and other parameters and apply the deltas to the object, deepcopying only once if mutate is at True ...
    Delta.apply_all(object, [delta, delta, delta, ...], mutate=True, ...)

    Design is hard and criticism is easy so I must say again : DeepDiff is awesome.

    As for the PR that's great !!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions