Preventing API Errors in Parabola Flows

November 18, 2021

I've worked on dozens of Parabola flows that depend on accessing data via API, expanding the JSON results, and then taking steps on the resulting columns.

But, one problem that pops up with frequency is when, for any reason, the data that the API is supposed to return either comes back empty or incomplete.

That breaks the whole rest of the Parabola flow. What we need is some error handling.

Fortunately, Parabola offers a workaround:

Instead of using the "Pull from API" step, use the "Enrich with API" step to access APIs. The Enrich step is more robust than the Pull from API step in a situation where, for any reason, data doesn't come back from the API request. Here's why:

  • In a "Pull from API" step that comes back empty, you have 0 columns and 0 rows. Because of that, no future steps can act on the data coming from the "Pull" step. Everything breaks.
  • In an "Enrich with API" step that comes back empty, you already have columns from prior steps, and you could add a couple extra steps that would conditionally create the columns the API is supposed to generate (via JSON that gets expanded).

For example, say you are trying to scrape a site using ParseHub (PH). Calling your last run of a PH project will return JSON with the contents you scraped. 

But, what happens if it comes back empty, or without all the JSON you were expecting? (This randomly happens sometimes)

Everything breaks and your flow will fail:

Because there's no data, you can't add backup columns after this.

If instead, we start with a system that uses the "Enrich with API" step, we might have something like this:

  • A Google sheet or CSV with two columns: The project name and the PH project ID.
  • An Enrich with API step that uses that ID dynamically in a GET request that should return data.
  • If it doesn't return data, you actually could add backup columns in subsequent steps.
  • So, we add an "Insert Text Column" step with dummy JSON, or JSON from a previously successful run, which the rest of the flow will use (and therefore not break).

Here's what the steps look like when you have a working Enrich with API step:

If no data is returned, subsequent steps will break.

So, we add the "Insert Text Column" step with dummy JSON and name it the same thing as the desired API response. In this case "api.jobBasicsDiv". Crucially, we set it up so that this column is only created when that column doesn't already exist, ie, when no data was returned.

With this set up, as long as your dummy JSON contains all of the same fields as the JSON your API is supposed to return, your subsequent steps will work.

Then, you could simply filter out the contents of your dummy JSON in later steps, before you export/send it anywhere. 

You could apply the principle of using "Enrich with API" instead of "Pull from API" to any scenario, not just ParseHub. It's how I intent to build all of my flows for clients going forward.

👋 Looking to no-code freelance?

I'm putting together a list of companies that could use some help.

The list is full of detailed info, but I don't want these companies to get overwhelmed by pitches, so only a limited number of freelancers will get access.

Put in your email below and I'll send it your way.

3 left at free price tier. Next price: $15