Personal.X-Istence.com

Bert JW Regeer (畢傑龍)

Editing SQL data dumps

I must enjoy pain on a huge level, as I am still using MySQL, and even worse, I am still using software that is using reserved words in MySQL, so doing a dump from the DB and then re-inserting it will cause some funny problems.

Now, since that will fail because of the reserved words I have to manually edit the field names, and add the proper quotation marks around it, so MySQL will grok it. I have a multitude of editors available to me: TextMate, SubEthaEdit, nano, vim and a few others.

I try to open it in SubEthaEdit. It goes on it's merry way and allocates a gig of ram, and then fails miserably. Same thing happens with TextMate. These programs were designed to do text editing on text files, with syntax highlighting.

nano causes the system to lock up while it tries to allocate about 1.10 GB of ram before it too is cut off by the kernel. This is insane, text editors that are unable to deal with a relatively small dump. Only 600 MB worth, I have done dumps where the data was well over 16 GB.

So after all this, I open it in the application I knew could handle it, but hate to use since it's syntax is really still baffling to me. vi.

vi uses 10 MB of memory, and has the file open and ready to be used within a few minutes. Searching through the file is just as fast as using grep or other tools. The makers of all the other text editors should take a page from vi's book, and figure out how they got so much functionality into such a small package, and off course being able to open multi-GB files is always a big plus for people that have to deal with such things!