At $dayjob one of our Django application used to have PickleFields as model attributes. They where mostly used to store metadata in the form of dicts. But as Pickling is considered a security risk we wanted to get rid of them. Since all information would have to be translated to JSON for the API anyway it made sense to store it in JSON as well. This is easy using the implementation provided by the JSONField.
Since we use South in our application changing the field type was easy.
Then creating the migration script.
However this won’t actually convert the data stored in the database to the new format. So with every request the result would be:
As the raw database data was a base64 encoded Pickle, which clearly is no valid JSON.
One approach to solve this issue would have been to:
- create a schema migration to rename the Pickle field type attribute to a temporary name.
- create a schema migration to re-add the attribute with a JSON field type.
- create a data migration to read the temporary Pickle field values into the new JSON field.
- create a schema migration to remove the old Pickle field type attribute.
But as I already had pushed out this code I wanted to know if there was another solution. After some research I found just that.
With the Django ORM it is possible to perform raw queries and redefine the column names. So you could do something like this:
Since the model doesn’t have a definition for _attribute_old _it will return the raw database value for that record.
Using this method the following data migration can be created:
And the reverse can also be made: