Friday, May 21, 2010

Importing Transactions into Mint

Mint is a very nice financial management tool. Unfortunately it lacks an important feature, the ability to import transaction information from a text file. If your financial institution is not supported or if you just want to input a bunch of old transactions you are pretty much screwed, since the only option is to it manually through the UI. Like every good programmer I'm extremely lazy, so there was no way I was going to spend hours and hours mechanically clicking buttons. It took me sometime but I created a way to automate it. It's hacky and ugly, since I put it all together in less than 2 hours, but it works and saved me some time, so I thought I would share it with you guys.

Since the data inputing can only be done though the UI I turned my head to Browser UI Automation. A quick search on stack overflow turned up Selenium, an open source suite of software to record and run Browser UI test cases. It is composed of:
  1. A java server that automates Firefox and can run test cases.
  2. A .Net library that can can communicate with the server, thus automating Firefox from C#. (There are libraries for numerous other languages as well, from Java to Ruby)
  3. A Firefox plugin, that you can use to record a set of browser actions as a test case. It can also export the test case as C# code that uses the library from above.
I came up with the follow strategy to use Selenium to solve my problem:
  1. Record a Selenium test case of me logging into mint and then creating a transaction. I needed to install the Selenium Firefox plugin before doing that.
  2. Export the test case as C# code.
  3. Modify the Selenium code so that the mint user name and password are received as parameters and the transaction create code is in a loop that runs multiple times.
  4. Create a VSTO Excel spreadsheet and embed the Selenium code in it. Add references to it to the Selenium assemblies. Have cells for the username, password and data to be imported into Mint. Have a button that calls code that instantiate the selenium test case class and runs it.
To use the spreadsheet you should do the following:

Prereqs:
  1. Firefox 3.x with the Selenium Firefox plugin installed.
  2. .NET framework 4.0
  3. Excel 2010 (Excel 2007 should work as well but I have not tested it.)

Procedure:
  1. Install the spreadsheet, clicking on setup.exe
  2. Open the spreadsheet and fill the username and password.
  3. Paste the transaction data. The columns here are transaction date, name, category and amount. The values here MUST be exactly what you would type into mint, or else all hell will break loose. The only exception is the date field. If Excel recognizes it as a date, it will work. Notice that this probably mean that you will have to massage you data somewhat.
  4. Open the selenium server. Its a Java executable called "selenium-server", on the same folder as the spreadheet
  5. Click run
The you just have to sit back and watch the thing run. It is not that fast, It creates about a transaction per second or a bit less, but that sure beats entering them manually. I've also seen a selenium call fail for no good reason. If this happens use the Firefox window to look at Mint and figure out the transaction that was being created when the failure happened and restart the process from that transaction.

Here's the "executable". It includes the Excel spreadsheet and everything you need to run it including the selenium-server and dlls, bur excluding the prereqs noted above.

And here's the source code. Once again, includes all the dependencies.


Let me know if you find it useful! :-)

David

Update 1: Updated the executable file with a properly "published" xlsx file. The older file would not run because of security problems.

10 comments:

  1. great, great idea. but your links are broken. :(

    ReplyDelete
  2. Sorry for that. I was misunderstanding how sky drive public files work. It should be fixed now.

    ReplyDelete
  3. Hi - I am getting an error when opening the spreadsheet: The common language runtime could not be loaded by E:\Documents and Settings\brian\Desktop\MintTransactionCreator\Release\MintTransactionCreator.xlsx. Contact your administrator for further assistance.

    The .net 4 framework is installed. Any ideas?

    ReplyDelete
  4. Attempting to open MintTransactionCreator.xlsx results in an interminable "File conversion in progress" notice in Excel 2003 with the 2007 file conversion plug-in installed..

    ReplyDelete
  5. OS: Windows XP SP3
    MS Office 2010 Trial - Excel 2010
    MS .NET Framework 4.0 Client Profile
    MS .NET Framework 4.0 Extended

    The xlsx file loads but then displays the error message: The following solution cannot be loaded because microsoft office support for the .Net framework 4.0 is not installed: xxxx.xlsx

    Is there anything further I can do?

    ReplyDelete
  6. Managed to get everything up and running. But everytime I run it, it logs in fine, inserts the first transaction fine, then fails part way through the second transaction. Any thoughts?

    ReplyDelete
  7. Hey all who posted. For those who has problems setting the thing up. Some possible things to look at:
    -Have you downloaded the setup, rather than the source code?
    -Have you installed Office 2010 with the default option, rather than leaving out components.

    For those that got it to work but are facing problems mid transaction creation. My best guess is that Mint's UI somehow changed and broke the selenium code. Can you give more detail on what happened? I.e. at what step the process stopped and what were the symptoms of the problem? Another possibility is that the second line of your dataset had invalid information.

    ReplyDelete
  8. Hey David,
    Very cool. Do you happen to know the C# code required to automatically log-in to Mint and download the complete transaction CSV file? I'm trying to automate it in an SSIS package in SQL to better view the data.

    ReplyDelete
  9. Dude, thank you so much for writing this program. Unfortunately, both of your links no longer work.

    ReplyDelete
  10. Thanx. I figured out an easier way using Firefox extension iMacro. See my comment here

    http://aaronfrancis.com/blog/2013/4/11/importing-transactions-into-mint

    ReplyDelete