LunchLine

Table of Contents

Purpose

LunchLine was born out of a simple desire to split a large transaction into accurate splits with minimal effort.

If you read my first post about Lunch Money, then you know that I’m a big fan of the flexibility and simplicity of Lunch Money. However, as with all things, it’s not perfect. Luckily, the solution for simple tools is to build an open API, which is exactly what Lunch Money has done! To be fair, Lunch Money would have to build a ton of additional functionality (with potential privacy implications) to make a feature on parity with LunchLine.

The Problem

When you go shopping at Walmart, Kroger, Target, Amazon, etc, you may spend a large sum of money. Then when you go to classify this large sum of money, you typically only have the option to utilize a single Category. i.e. Groceries, Health, or Definitely-Not-Another-Piece-Of-House-Decor 🤡.

But more often than not, you’re spending money on a variety of things. Lunch Money does allow you to split your transactions into multiple categories, but it’s a manual process. If you spent $300+ dollars at Walmart, you’re going to have to set aside 15 minutes or so to manually go through a categorize everything. To make things worse, many times the receipts aren’t very descriptive.

Do you know what MST FMT G 18AE means? I don’t.

But first, photos

Before anything else, how about some pictures to show what this tool does?

The first step I usually do, is run lunch check in my terminal to see if I’ve got any unreviewed transactions in Lunch Money.

This command will show all unreviewed transactions, even if I don’t have a rule that would run on them. For example, I don’t have a rule for Publix in the image above. But still, it’s nice to just get a quick overview of what needs reviewing.

Next, I’ll typically run lunch split --just-one which allows me to interactively pick a single transaction to split. As I type, the list will filter down to any matching property which contains the string I’m typing.

When I find the transaction I want to split, I can hit enter to select it. And the script will take it from there.

When the script is done gathering the receipt data for the given transaction from that retailer, it will auto-categorize everything it can from the keyword data. (More on that later) Anything that doesn’t match, will be presented to let me manually categorize it. Then it asks me for an optional keyword so that it will auto-categorize this item in the future.

Apparently I’ve never bought cucumbers before?

After this process is done, LunchLine will automatically split the transaction into multiple transactions, and then post those updates to Lunch Money. It prints a small summary to the console, and then I can see the updates in the Lunch Money UI!

The final summary which shows that a non-trivial portion of our “grocery” trip was actually home supplies.

I also apply a LUNCHLINE tag on the transaction so I know this one was processed by the CLI. This is useful because I also have a tool for “reverting” my changes and restoring a transaction to its prior state.

The Solution

Since Lunch Money allows you to split transactions via the API, the only hard part would be getting the receipt data in a reliable manner. Indeeed, this is basically the entire complexity of this project—each retailer has different ways to get receipt data, they provide the data is varying data formats, and you can never get that data without logging into an online account.

The System

Since I’ve got a fair bit of experience with browser-automation, I decided I’d try to automate logging into various retailers using Puppeteer, in order to get my receipt data. However, as it turns out, this is a lot harder than it sounds. Most modern retailers have insane online security that means even a working solution, will get flagged as a robot from time to time.¹

The general approach

To handle the various implementations that would be required for each retailer, I structured each retailer as a “rule”. A rule took the following structure:

export type Rule = {
    name: string;
    run: (t: Transaction) => Promise<boolean> | boolean; // Returns whether or not to keep running other rules
    matches: (t: Transaction) => Promise<boolean> | boolean; // Returns whether or not the rule matches the transaction t
};

This gave me a general interface by which each rule could be run. When a transaction was added to Lunch Money, I’d run it through the list of activated rules, and the first one that passed the matches function, would be run. Eventually, it might make sense to allow running more than one, but this was simple and works for my use cases.

Then the script would run the run function. Here’s an example of the rule for run function for Target. I’ve annotated it with comments for clarity.

async function run(transaction: Transaction) {
    // If we've already got cached receipt data for this transaction, skip the browser automation
    let rawTargetData = await getTargetReceiptFromDisk(transaction);
    if (!rawTargetData) {
        // `director` is a convenience wrapper around Puppeteer that gave me some useful features without needing
        // to configure them in every single rule. For example, I made use of network spies so that I didn't have
        // to scrape HTML for data--I could just listen for the network request that contained the data I needed.
        const director = await initDirector({ slowMo: 20 }, configureSpies);
        const { page } = director;

        // Target-specific login function
        await login(page);
        await navigateToOrderTabs(page);

        // Target-specific function to save the receipt data to disk
        await saveTargetOrdersToDisk(director);
        // Target-specific function to decorate the receipt data with line-item specific information such as per-item
        // tax, quantity, and desription.
        await decorateOrdersWithDetails(director);

        await director.closeBrowser();
        // Load the data we just saved to disk
        rawTargetData = await getTargetReceiptFromDisk(transaction);
    }

    if (!rawTargetData) {
        console.warn("Transaction:", transaction);
        throw new Error(
            "After fetching fresh transactions from Target, still could not find matching receipt"
        );
    }

    // Here I convert retailer-specific data formats into the Receipt interface that worked with the generalized categorization
    // and splitting logic.
    const receipt = mapRawTargetDataToReceiptType(rawTargetData);
    const splitMap = await convertLineItemsToSplitMap(receipt);
    // Though not financially accurate, I just split tax evenly across every category.
    spreadTaxAcrossSplits(splitMap, receipt.tax);

    // Send the splits + categorization information to Lunch Money to split the transaction
    await LunchMoneyAPI.applySplitToTransaction(transaction, splitMap);
    return true; // Return true that we want to stop running other rules
}

As you can see, there’s a lot retailer-specific logic within this function, and my general design goal was to keep all of the retailer-specific logic, in these functions and generalize as much as possible without compromising readability.

The categorization happened via an on-disk JSON file that mapped categories to arrays of keywords. Here’s an example of our “GROCERIES” category:

{
    "GROCERIES": [
        "Onion",
        "Banana",
        "simple truth",
        "cookies",
        "raspberries",
        "chicken broth",
        "marinara sauce",
        "avocado",
        "cascadian farm",
        "dave's killer bread",
        "ice cream",
        "pork loin chops",
        "blueberries",
        "boar's head",
        "spice world",
        "sure-jell",
        "kombucha",
        "hot sauce"
    ]
}

The categorization logic is VERY simple. If the line item description contains any of the keywords (case-insensitive), within a given category then it’s categorized as that category. This is obviously not perfect, but it’s good enough for my use cases. I thought about eventually getting fancier, but after ~2 years of using this, I haven’t had a strong need for anything more complex.

The End Result

In the end, I built rules for:

• Walmart
• Target
• Amazon
• CPWS (Local Utility Company)
• Venmo
• Kroger

Most of these solutions work perfectly. Some keep giving me issues with regards to not letting me automate logging in to my account without getting flagged as spam. In response, I’ve actually pivoted to a slightly different method of gathering the data, which I plan on writing about in the future (because as of this writing, I’m not done yet).

The new approach gets around authentication by using http requests between a Chrome extension and a small node http sever in the CLI. This allows the browser to talk directly to the CLI without needing any automated logging-in. There are some trade-offs that I’ll discuss in a future article.