Shortcut.ai: The AI Excel Intern

I really hope that when I look back on this three years from now, I’ll realize how limited my perspective was.

I’ve been on the lookout for an AI solution to the tedious task of spreading a historical financial model. I actually like spreading financial models - I feel like I understand the business a lot more just through osmosis. But your boy is lazy sometimes. I thought, how hard could it be? After all, the financials are all public on the SEC.gov website and formatted in standardized fashion. It turned out whether you use AI or mechanical code, it is not a trivial task. I’m not a strong coder or an expert prompter, but I know enough to see that the inconsistencies in the results point to a deeper problem than just more rules/conditions or generalizations.

When the Excel-native AI assistant Shortcut.ai launched with an awesome demo video, I was intrigued. A tool that lives inside Excel? It can create a model from scratch in minutes?

And then I got the invite. Imagine my excitement.

I was on a family road trip to the Grand Canyon when Nico, Shortcut’s founder, teased out 100 early invite codes on Twitter. I was on our way to see the sunset at Hopi Point when I saw the post. I dropped everything, told my kids to go touch some rocks, and signed up for the service.

This was it. I put my hands together and whispered a prayer as the sun set over the Grand Canyon. The moment was transcendent… not the canyon, but Shortcut.ai. The canyon was fine, I guess. Just a big hole…not the best hole I’ve seen either.

I opted for the $40/month plan. There’s a $200/month tier, but I’m a value investor, which means I’m pathologically frugal.

Time to test.

---

The Only Test: a Historical Financial Model

If I could get Shortcut.ai to put together an accurate historical financial model for a company, I would be happy. Not only because it would save me a lot of time, but because I would be assured that this tool is developing in the right direction. Everything is built off of the historical financials.

A week or so ago, I asked Shortcut.ai to generate a 10-year historical income statement for Nexstar (NXST). This should be a pretty simple task. The initial result:

hmmm….lots of missing detail, but the format is pretty neat.

I asked Shortcut.ai about the missing data - do you see that? Can you populate the missing data? Shortcut.ai came back with this:

That looks better. At least we got some data, but what happened to the data from 2015 to 2020? When I examined the numbers closely, I got some rude awakenings. Most of the numbers don’t check out… Some were close; others were invented.

I’ve seen this before. It’s that uncanny ChatGPT confidence: sounding right, looking right, and being wrong.

I enter the exact prompt into ChatGPT and lo and behold I got similar information, except in a crappier formatting. Interestingly, ChatGPT also didn’t provide me with the 2015 to 2020 data. It seems to me like Shortcut.ai is prompting ChatGPT and packaging the answer in a prettier format. This was confirmed when I went back to review the steps that Shortcut.ai took. It searched for the financial information and scrape it off of websites, but it didn’t actually go into the SEC filings.

I asked shortcut.ai to do a few other things which it did very poorly on. I won’t go into detail on them cause if shortcut.ai can’t get the basics right, I don’t know how usefully it will be for other tasks.

$40 bucks down the drain, I guess. Not cool but not unexpected given my experience with AI tools for financial reporting.

This morning, when shortcut.ai officially launched, I went back to the website and was pleasantly surprised by some improvements. For example, when I asked the same shortcut.ai to complete the same task, it came back with more data that look more directionally right. Instead of scrapping off of websites, it looks like Shortcut.ai actually downloaded the SEC filings. There’s what the initial prompt resulted in.

Still a lot of missing data, but I’m encouraged that shortcut.ai is actually downloading the actual SEC filings.

I asked shortcut.ai to fill in the blanks, and it returned the following results.

Looks good, right? At least better than before. But once you look at it in more detail, you will notice that the balance sheet has places that don’t balance, and the income statement/cash flow statements have places that don’t sum up correctly. Some numbers are close and some numbers are wrong...

Just for HEE-HAs, I asked shortcut.ai the exact same request again, and it gave me something that looked completely different, including formatting and line items. It even thought 2024 was in the future and gave me a projection for that year, how thoughtful!

I also asked shortcut.ai to provide me with a quarterly historical financial model. It proceeded to divide the annual numbers by four and populate the quarterly columns.

---

How the Cheapo Version of Shortcut.ai Works Currently

I’m only using the $40/month version of Shortcut.ai, so maybe the $200/month version can do a better job - but I don’t think I want to throw another $160 down the drain to find out that it’s just a prettier version of ChatGPT. Also, since we’re dealing with AI, it’s very possible your experience is different than mine, ha!

Shortcut.ai, I suspect, is built on top of ChatGPT and/or other LLMs - using it to generate content, and layering formatting, user experience, and financial context on top.

Which is smart, and honestly what you would expect. Build a nice wrapper and wait for LLM to get better dealing with financial data. The core competency of Shortcut.ai is never to be better than the large tech platforming investing billions of dollars building foundational models. It is in building a good user interface and locking in the analyst work flow.

Unfortunately for all of you guys hoping for a miracle, once you peel back the layers, the core weaknesses of LLMs are still there. They don’t compute and verify. They infer and guess. The biggest problem is inconsistency in results - everytime you ask the same question to an LLM, you get a different answer. It’s nondeterministic by design, which is not great if you’re a financial analyst.

---

But Are We At Least On the Right Trail…or Halfway Down the Canyon With no Water and a Broken Headlamp?

Shortcut.ai is beautifully designed. The promise is real. And I genuinely hope it evolves into the thing I wanted it to be. But can AI in its current LLM form achieve this dream?

ChatGPT and Grok are amazing at language. They write memos, emails, and pitch decks better than most humans. But when it comes to quantitative work - parsing filings, scrubbing models, reconciling financials, they start breaking down, or even worse, straight up lying.

I have real reservations about the ability of LLM to deal with quantitative tasks. The fundamental issue is architectural: LLMs are trained to predict the next word, not the next cell value. They're great at capturing and regurgitating ideas/tones - because human language is vague and forgiving. Spreadsheets are none of those things. In modeling, precision isn’t optional. The more precise we want a task completed, the less trustworthy an LLM is. Maybe a combination of LLM and mechanical coding will get us close. So far, I feel like we’ve been fooled by LLMs and the ambiguity in our language.