September 24, 2020

A brief history of web app automation: From Yahoo Pipes to Zapier to automation everywhere.

How automation took over software, and the world.

by @AndrewJDavison

Zapier Certified Expert and Integromat Partner running Luhhu - we do cool automation stuff.
AndrewJDavison's avatar

It used to be that automation was limited to factory floors and robot product lines. Now, automating business tasks is sweeping across every market — and fast!

In 2020, just about everyone has used — or at least heard of — apps like Zapier, IFTTT, and Integromat, or at least used built-in automation tools and workflow builders inside apps like Slack. For many businesses, automation is a non-optional investment. If they want to compete in their industry, then automation is a must.

But how did we get there? How exactly has business process automation evolved? When did it take over the workplace? And what’s next, now that seemingly every app has automation built-in?

The early pioneers of business automation software

In the beginning, the only way to automate workflows across multiple apps was to write your own code to integrate manually. That was only possible for software that included an API, far from a guarantee in the early days of web apps. Even when two apps could be integrated, the options available were so limited that it often wasn't worth the effort.

Still, some companies saw the potential. And now, as is often the way, it seems obvious that software automation would be as viable and successful as industrial automation.

But that hindsight was only made possible by a handful of pioneers.

IBM’s WebSphere Process Server

IBM WebSphere Process Server

One of the first of these pioneers was IBM, a company that has been at the forefront of more emerging technologies than just about anyone else. Automation is no exception; the tech dinosaur saw an opportunity for automation in the early noughties and launched the WebSphere Process Server in 2005.

The goal of WPS was to help businesses visually represent their workflows. It used a standardised notation, which made it easier to organise and communicate how certain processes were completed by the business.

This software eventually became automation-oriented, but never managed to find mainstream success. It was too technical, and — like many of IBM's products — it never made it past the enterprise market, where it still lives on as IBM App Connect today.

Still, it demonstrated that organic business processes could be digitised and, ultimately, automatised.

Yahoo! Pipes: The first major player

Yahoo! Pipes

Enter Yahoo. Back in the day, Yahoo! was a sector leader, with their namesake search and Yahoo Mail making it a popular consumer software company.

So in 2007, when Yahoo! Pipes launched, it quickly became a popular product, one some users still mourn today.

Pipes was a web app to create simple automations with a drag-and-drop interface. It was one of the first to make this process so visual, as well as one of the first to incorporate APIs.

The result was an online tool that pulled in data from various sites and web apps, then set up rules and processes tied to that data. While it mostly revolved around data aggregation and manipulation, it served as a template for later automation services, adding the mainstream appeal that WebSphere Process was missing.

A rapidly growing automation market

The web app automation market was beginning to really take off in the early 2010’s—even if no one realised it yet. Off the back of WPS and Pipes rose an entire market of sites and apps trying to be “the next big thing” in automation.

As WPS and Pipes started the web automation market, SaaS apps began to take over. AWS, Twilio, Stripe, and other services that formed the backend of today’s SaaS giants launched in rapid succession, just as tools like Google Docs and Sheets had gone mainstream enough for Microsoft to bring its Office apps to the cloud. Every new popular business app was a SaaS app, it seemed, with an API ready to automate.

Someone just needed to put all of these pieces together.

IFTTT: Automation for the masses

Original IFTTT description from their early site

IFTTT, short for If This, Then That, launched in September 2011, as one of the first API-powered automation tools most people would use. Just half a year after its official launch, IFTTT was already home to more than one million automations.

IFTTT was the first service to introduce a plug-and-play approach to automation at a massive scale. It connected apps that had previously been wholly separate, empowering people to create new automations based on each app's unique features. And it was free (with a generous free plan today, even after launching paid plans nearly a decade after launching), with a simple interface anyone could try it out.

“If this app does this, then this app does that,” IFTTT taught the world.

What set IFTTT apart was that it made programming logic as user-friendly as possible. Programming looks complicated, with its lines of code and fussy punctuation rules. At its most basic, though, programming is simple logic tools. "If this, then that", or "While this is happening, make this also happen", developers teach computers, and by combining that logic in countless ways, you get every piece of software in the App Store.

IFTTT showed that you can visualise programming concepts so that anyone can understand them, and use that as a way to bring the power of programming to the mainstream.

While IFTTT's popularity has been superseded by other services today — partly, and somewhat ironically, because it's too simple — it showed tech companies that customers are fast learners when you give them the right tools.

And, more pertinently, it showed that the world wanted automation. The time was right.

The rise and rise of Zapier

Zapier screenshot

"So I was thinking. A killer web app might be something that extends the functionality of other apps' APIs. The tagline could be: 'Extending the functionality of the web.'"

"Dude, that would be awesome. The API of all APIs."

"Very meta."

~ Bryan Helmig to Wade Foster, the prescient founders of Zapier.

In 2011, two jazz musicians from Missouri started a weekend project that would grow into a company with $50 million in annual revenue within its first seven years.

From the beginning, Zapier was a simple idea based on tools that were already there. Similar to IFTTT, it used pre-existing APIs to connect business software and make automation simple and ubiquitous.

Unlike IFTTT, however, Zapier allowed users to go as deep into the technical aspects as they liked — creating a platform that had far broader limits and possibilities than anything else on the market. IFTTT connected any two apps; Zapier let you connect dozens of apps into detailed workflows, with filters to let the tool make decisions for you. You could even add code, if you wanted. And where IFTTT focused on more consumer apps and IoT devices, Zapier focused on business software and workflows.

It appealed to a massive audience, from startups to established businesses, non-coders to developers.

Sparking the no-code revolution

The idea of replacing programming with a visual drag-and-drop interface harks back to WebSphere Process in 2005—and even to Apple’s HyperCard in 1987. Despite the numerous apps that tried to capitalise on this idea, it took the rise of SaaS, APIs, and Zapier’s automation to finally make the new no-code movement possible, where “citizen developers” could build custom tools to automate work, without coding.

Now, all entrepreneurs need is a solid business model, a handful of software to perform each function they need, and an automation workflow tool to tie them together, and they can have their digital business up and running in hours. More often than not, it’s Zapier automations that makes these new no-code apps possible.

And with such a big slice of the pie, it wouldn’t be long before Zapier’s authority was challenged.

Built-in automations vs Zapier and IFTTT

A built-in Slack workflow

When apps like Zapier and IFTTT were first launched, they were two of the only avenues for cross-platform automation.

Today, however, the story’s very different indeed.

There’s an increasing number of dedicated workflow automation apps, including Integromat, Workato, Amazon AppFlow, Microsoft Flow, and more. Then, more and more mainstream apps—including Slack, Airtable, and more—are entering the scene, with built-in automations and workflow builders as a simpler alternative to the complex nature of Zapier’s limitless platform. While most of these apps offer limited options, many people only need limited automation.

But will it stick?

For now, it looks like these built-in automations are just a convenient feature of the growing SaaS market. Even though popular apps like Slack, Pipedrive and Airtable have built automations directly into their platforms, it's unlikely that these will ever rival Zapier — unless these apps first convert to an automation-centric business model, too.

Until then, third-party solutions like IFTTT and Zapier will likely thrive. Especially among those of us who like a little challenge.

Automating your business in 2020 and beyond

All this talk of history and revolution has finally brought us, for better or worse, back to the present day.

If you're a business owner, automation can have a huge impact on your operations. It can cut costs, increase your productivity, and supplement a shortage of workers when you're first starting out.

If it seems intimidating, don't worry — we'll cover the first steps here.

Knowing when to automate

When you first start automating, one of two thoughts tends to pop up:

  • I can't automate anything — how do I learn?

  • I could automate everything — now what?

If you're struggling with the former, then it can help to look up tutorials and guides for apps like Zapier. This will not only teach you the basics, but you'll also start to get a feel for automation and spot opportunities in your own business.

If you're struggling with the latter, then the easiest way to decide if you should automate a task is to look at the cost vs. benefits.

Is automating this process cheaper? Will it lead to fewer errors? Is it simplifying things, or overcomplicating them? Will you understand how it works, or will it become a blackbox?

Choosing between Zapier and built-in solutions

Another choice you'll need to make is which platforms you're going to use. We've covered a few in this article, such as Zapier, Integromat, and IFTTT, as well as built-in automations in apps like Slack and Airtable.

Though you could simply throw automations onto any and all of these resources, chances are very few of you reading would like to embark on such a cluttered route.

If you're not sure whether an all-in-one platform like Zapier is right for you, or if you should opt for simpler built-in solutions, here's a simple flow chart:

Use built-in automations if:

  • You only want to automate in one app

  • You need something simple and fast

Use an all-in-one platform if:

  • You want to incorporate several automations into your workflow

  • You use a wide variety of apps

  • You want to go deeper into the world of automation

Finding the right tool for the job

If you do decide to use a platform like Zapier or IFTTT, you'll run into the problem of having too many options to choose from. At the time of writing, Zapier has over two thousand apps to choose from. As you might imagine, many of these apps overlap, offering similar features to one another.

Deciding which apps to use will largely come down to how many automation triggers and actions they offer, how much they cost, and how comfortable you are with them. In other words, visit the websites for the apps you're deciding between, get a feel for them, and determine which is right for you.

The future of automation

When Zapier first launched, it was largely a one-of-a-kind service. Today, though, it has stiff competition from companies like Integromat and Parabola, as well as the mainstream SaaS apps with built-in automations.

Each of these apps is taking a unique approach to automation.

Zapier aims to be the one-stop for combining any app with any other app, while Integromat has a smaller selection with more integration features. Parabola offers a data-centric approach, while built-in automations make things as simple as possible.

Regardless of which app is best for your business, the future of automation is exceedingly bright. The possibilities are endless, and there is a wealth of ideas within those possibilities that can help you capitalise on your company's success.

If it is a race towards automation, it’s a marathon — not a sprint.

Join the discussion: Do you prefer built-in automations, or automation workflow tools like Zapier and IFTTT?

Image Credits: Header photo by tian kuan via Unsplash. WebSphere Process Server screenshot via IBM. Yahoo! Pipes screenshot by Mark Woodman via Flickr. IFTTT screenshot from December 2011 via Wayback Machine.

alexhbass's avatar
@alexhbass (replying to AndrewJDavison )
2 years ago

Really great article buddy, enjoyed it a lot 😊

4 points
AndrewJDavison's avatar
@AndrewJDavison (replying to @alexhbass )
2 years ago

Thanks @alexhbass

1 point
AndyDentPerth's avatar
2 years ago

I'm really busy so this is a comment rather than an entire essay.

I'm disappointed in the amount of history skipped - this is an article about Web Era Automation as there were decades of automation tools prior to that.

The entire philosophy of Unix from very early days was connecting tiny tools with (text) pipes.

In the Classic Mac era, AppleScript was a low-code environment for end users and it provided a formal scripting interface into an entire generation of apps. You couldn't put a serious app out on the Mac without journalists checking if you provided scriptability.

As an example, I found myself consulting at KPMG to help them clean up an entire budgeting system they had built using FileMaker Pro and AppleScript. I worked for Australia's largest Real Estate Printing company (a horrifyingly busy category) to rewrite the system one of their operators had built in AppleScript. The rewrite was only because at their scale, they needed it to run a lot faster. The user-developed system worked very well, automatically generating web pages from incoming image folders.

Those were jobs in the mid-90's.

4 points
AndrewJDavison's avatar
2 years ago

Thanks, @AndyDentPerth - valid point there. If you were to really cover this subject you'd need a book rather than a blog post.
Interesting to hear your personal experiences here. Don't know anything about AppleScripts, but was it decent to work with?
Perhaps we need a pt2 to this post from you covering the pre-web era?

2 points
maguay's avatar
@maguay (replying to @AndrewJDavison )
2 years ago

Great point that this article focuses on web software automation—where there was desktop software automation for decades prior, and more recently native mobile software automation with tools like the iOS Shortcuts app.

I just edited the title to "A brief history of web app automation" to clarify that.

2 points
AndyDentPerth's avatar
2 years ago

I'd love to write a pt2 about the earlier history but between a recent day job, still-going startup and painting an entire house (exterior), I'm drastically short of time.
Dave Winer's Userland Frontier is another early scripting must-mention.
AppleScript was a great architecture in that it encouraged scriptability and discoverability of scripting apis. As a language, it suffered the same nearly English problem of HyperTalk. Sometimes it was frustratingly difficult to write but always easy to read.

2 points
maguay's avatar
@maguay (replying to @AndyDentPerth )
2 years ago

If you ever manage to have the time, I'd love to read and help promote that!

There's an interesting point there on the difficult of writing near-English scripting languages that mirrors the issues new devs face in coding and using Terminal: Discoverability. It's something that fuzzy matching has helped solve to a degree in command palettes and search tools like Spotlight/Alfred, and it'd be interesting to see a modern update to Automator/AppleScript combined with autocomplete and discovery tools to aid learning what you can do with it.

2 points
kespears's avatar
@kespears (replying to AndrewJDavison )
2 years ago

Great post Andrew. I am a newbie and this article was very helpful.

3 points
AndrewJDavison's avatar
@AndrewJDavison (replying to @kespears )
2 years ago

Thanks @kespears!

1 point
NBNite's avatar
@NBNite (replying to AndrewJDavison )
2 years ago

Very thorough and well-written piece @AndrewJDavison
I can't imagine how much time goes into these pieces and your sharing this here is much appreciated!

2 points
AndrewJDavison's avatar
@AndrewJDavison (replying to @NBNite )
2 years ago

Thanks for the kind words @NBNite!

1 point
iCanAutomate's avatar
@iCanAutomate (replying to AndrewJDavison )
2 years ago

Great job with this post Andrew! I especially like the consideration between native integrations and integration platforms. I've been telling people that if a native integration can do the job, just use it rather than building a custom one using Zapier or Integromat.

2 points
maguay's avatar
@maguay (replying to @iCanAutomate )
2 years ago

An interesting twist on native vs. 3rd party is how Slack's workflow builder now directly connects to Zapier, which lets you envision a future where the simple, in-app stuff is handled by Slack Workflow Builder while 3rd party integrations are left to tools like Zapier.

Or, it's a race of who can innovate faster, where the built-in integrations could be the fish that eats the whale.

2 points
AndrewJDavison's avatar
2 years ago

Thanks @iCanAutomate!
And yes, native vs. 3rd party... it's best to work with the native option so long as its technically mature and stable.

2 points
What's the best way to get feedback about a design or product?

Do you have a favorite survey or poll tool to get feedback, or a unique process to get actionable insights from your users about your design, product features, pricing, and more? There was an inte...

What's the best tool for building a customized API documentation?

Let's say you have an API you want to document and you want to: - Let your users test their calls to the API thanks to a "playground" like the one on the Figma API - Be able to customize it to matc...

The community for power users.