I recently had to switch some caches from Memcached to Redis, and was trying to figure out how to best achieve that on a live production server.
On a staging server we would’ve just deleted the database, deployed the latest service version, restarted the service and re-imported the data using the service API. That would’ve automatically created the required cache entries in the new Redis cache. Obviously in production environment that wouldn’t really do.
Since the Memcached server didn’t contain the cached entries any longer (fat-fingered /etc/init.d/memcached restart was to blame), I had to reconstruct the cached data from our services using the RESTful APIs they provide.
I usually do these sort of things either with scripting of some sorts or editor macros, whichever makes more sense. This time it looked like scripting would be most efficient.
So how would I manipulate JSON output from a script? I didn’t really want to start parsing it using some sort of JSON parser, so I looked into command line tools for doing that. Turns out there is an excellent tool called underscore-cli (at https://github.com/ddopson/underscore-cli) that does exactly that. It uses Node, and underscore.js for the hard parts.
It worked beautifully and fast. I extracted the information I needed from the JSON output acquired from our production service, then massaged it a little bit with some regular expressions using Perl and finally created the new cache entries in Redis using redis-cli.
|
1 2 3 4 5 6 |
$ curl http://prodserver/getall.json > data.json $ cat data.json | underscore pluck property_name > data.file $ perl -pi -e 's/\s+"(\d+)"/$1/g;' data.file $ perl -pi -e 's/,//g;' data.file $ perl -pi -e 's/(\d+)/set "redis-key-$1" 15/g'; data.file $ redis-cli -h prodserver < data.file |
Happy times.