You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Any None value make Delta non safely serializable #224
Describe the bug
If a data structure contains a None value the serialize/deserialize roundtrip breaks
To Reproduce
Delta(Delta(DeepDiff({"a":None}, {"a":1})).dumps())
deepdiff.serialization.ForbiddenModule: Module 'builtins.type' is forbidden. You need to explicitly pass it by passing a safe_to_import parameter
Although there is builtins.None in SAFE_TO_IMPORT, pickle need to use builtin.type for reason I did not figure yet
This works : Delta(Delta(DeepDiff({"a":None}, {"a":1})).dumps(), safe_to_import={'builtins.type'})
=>
<Delta: {'type_changes': {"root['a']": {'old_type': <class 'NoneType'>, 'new_type': <class 'int'>, 'new_v...}
But of course allowing builtins.type opens the insecure floodgates wide open, since any class can then be instanciated.
OS, DeepDiff version and Python version (please complete the following information):
OS: [Manjaro]
Version [rolling]
Python: 3.9.1
Additional context
Deepdiff and delta are amazing ;)
Side note : the documentations at https://zepworks.com/deepdiff/5.0.0/delta.html#delta-safe-to-import-label seems incorrect delta = Delta(t1, t2, safe_to_import={'mypackage.mymodule'}) implies that the safe_to_import parameter acts at Delta creation time (and it looks like DeepDiff syntax) and it is not.
It acts (and that's logical) at de-serialization time : delta = Delta(delat_dumps, safe_to_import={'mypackage.mymodule'})
Maybe Delta init does a little too much magic, Delta.from_diff() and Delta.unserialize() or other explicit classmethod whould make use more "intuitive".
@Annakan Haha thanks. Happy to hear you like DeepDiff and Delta.
Yep the safe_to_import is just for the import time. I will update the docs. And I agree that the init does too much magic. I just wanted the user interface as simple as possible which goes against being explicit. We can probably have some factory method exposed for people who want it explicit. What do you think? Since users have already started using the current api, I don't want to make any breaking changes to the api.
Also here is a PR to fix the issue you brought up: #225
I think the ClassMethod/Factory method would be an improvement.
And while you are at tit ... moving the code from the delta.add into a "proper" classical method like apply_to(object, mutate= ...) and having add calling this method would also make the interface more clear especially when mutating in place :).
For instance having :
my_delta.mutate=Truemydelta+my_object
looks a bit weird and takes two steps, and we need to be careful of + parameters order.
Instead ```my-delta.apply-to(my_object, mutate=True) looks more natural in that case.
And having a non commutative + is counter intuitive for many people given that all the base types that implement add are, unless I am mistaken.
Also maybe mutate does not really belong in the Delta, it is more a use time decision rather than a creation time decision to me.
For instance, In my use case I deepcopy once the source object and then apply a string of mutating Deltas rather than having each Delta deepcopy a big structure.
Another idea for a Delta ClassMethod apply_all which would take and object, an iterable of deltas and mutable, and other parameters and apply the deltas to the object, deepcopying only once if mutate is at True ... Delta.apply_all(object, [delta, delta, delta, ...], mutate=True, ...)
Design is hard and criticism is easy so I must say again : DeepDiff is awesome.
Activity
seperman commentedon Jan 14, 2021
@Annakan Haha thanks. Happy to hear you like DeepDiff and Delta.
Yep the safe_to_import is just for the import time. I will update the docs. And I agree that the init does too much magic. I just wanted the user interface as simple as possible which goes against being explicit. We can probably have some factory method exposed for people who want it explicit. What do you think? Since users have already started using the current api, I don't want to make any breaking changes to the api.
Also here is a PR to fix the issue you brought up:
#225
Annakan commentedon Jan 19, 2021
Thanks for your prompt answer.
I think the ClassMethod/Factory method would be an improvement.
And while you are at tit ... moving the code from the delta.add into a "proper" classical method like apply_to(object, mutate= ...) and having add calling this method would also make the interface more clear especially when mutating in place :).
For instance having :
looks a bit weird and takes two steps, and we need to be careful of + parameters order.
Instead ```my-delta.apply-to(my_object, mutate=True) looks more natural in that case.
And having a non commutative + is counter intuitive for many people given that all the base types that implement add are, unless I am mistaken.
Also maybe mutate does not really belong in the Delta, it is more a use time decision rather than a creation time decision to me.
For instance, In my use case I
deepcopyonce the source object and then apply a string of mutatingDeltas rather than having eachDeltadeepcopy a big structure.Another idea for a Delta ClassMethod
apply_allwhich would take and object, an iterable of deltas and mutable, and other parameters and apply the deltas to the object, deepcopying only once if mutate is at True ...Delta.apply_all(object, [delta, delta, delta, ...], mutate=True, ...)Design is hard and criticism is easy so I must say again : DeepDiff is awesome.
As for the PR that's great !!!