May 19, 2020

Computer Assisted Intuition: How Roam Research fulfills the web's original dream

Links were always supposed to go both ways. It just took decades to get there.

by @maguay

Writer. Amateur photographer. Information architect. Frm. founding editor @racket and @capiche, senior writer @zapier, Mac and Web editor @AppStorm. Personal blog: @techinch.
maguay's avatar

“The human mind does not work that way,” wrote Vannevar Bush of filing cabinets and alphabetical organization, in his seminal 1945 essay As we may think. He might have as easily invoked today’s computers, 75 years of progress notwithstanding.

Of all the metaphors computers could have used for data, the most familiar endured. Paper and folders, letters and business cards, desktops and trays. We recreated the office environment on a screen, and brought their physical limitations into the digital world.

Bush at the cusp of the computer age envisioned a different future. A researcher in this imagined world first “runs through an encyclopedia, finds an interesting but sketchy article, leaves it projected. Next, in a history, he finds another pertinent item, and ties the two together. Thus he goes, building a trail of many items.”

That’s the idea that brought us the internet, wikis, and today’s connected world. But it took a notes app to finally build the trail as originally envisioned.

A trail of thought.

Wikipedia graph visualization by Chris Harrison

The entire web could have been Wikipedia, if Tim Berners-Lee’s earliest ideas had worked out.

It started with Enquire within upon everything, an early encyclopedia that taught a young Tim “everything from how to remove clothing stains to tips on investing money,” remembered Berners-Lee decades later. “Not a perfect analogy for the web, but a primitive starting point.”

That provided the inspiration for Enquire, a notes app of sorts Tim Berners-Lee built at European research lab CERN as an early precursor to the web.

Links were central to Enquire. Instead of individual notes, Enquire’s notes linked to each other. You couldn’t just make a new note; you had to start by linking something in an existing note. A note about a colleague might link to their department and research documents, say.

Then, in a blend of bibliographies and cross-references, Enquire pages also listed everything that linked to them. So when you linked a colleague’s note to their department, “an internal link would appear on both,” Berners-Lee said. The trails that linked everyone to each other would start to appear.

Enquiring minds.

Links, after all, aren’t a word invented for tech like smartphone or CPU. They’re anchored in the real world, links between text like the metal links in a chain.

The earliest tech visionaries thus imagined links as two-way connections, the same way you see a chain on both the boat and the anchor. The brain, as Bush had written decades earlier, “operates by association. With one item in its grasp, it snaps instantly to the next that is suggested by the association of thoughts.”

You don’t come up with new ideas. You build on what came before. One idea combines with another which reminds you of something else you’d heard before, and eureka!, an idea is born.

If computers could aid innovation, they’d need to help us make those connections.

Ted Nelson’s Project Xanadu two decades after Bush’s essay imagined two-way links giving us a digital brain. As Nelson imagined the future, every document would be linked to its sources. You wouldn’t simply quote another document; you’d embed the quoted section from its source. Both your document and the document you quoted would show the connections.

“Computers could become much more powerful if they could be programmed to link otherwise unconnected information,” thought Tim Berners-Lee in the 1980’s while dreaming up what became the World Wide Web.

Links made that happen, but half of the original idea got lost along the way.

Just one new thing.

Tim Berners-Lee's WordWideWeb recreated online for CERN

Sometimes you have to cut scope to ship a project, simplify a concept to convince people to use it. Thus it was with the World Wide Web.

Tim Berners-Lee envisioned a world of websites about everything imaginable. His employer, the CERN laboratory, had more pedestrian concerns of managing the people and ideas under its roof.

“Our first target, humble beginning that it was, would be the CERN telephone book,” Berners-Lee decided after failing to get much interest in the first version of his WorldWideWeb browser.

And so, a few ideas were cut. Bi-directional links were out; “so long as I didn’t introduce some central link database, everything would scale nicely,” he theorized, so every page must stand on its own.

Also on the chopping board was editing. Tim’s ideal web made it as easy to edit web pages as to view them. “Simply being able to read the documents was good enough to bootstrap the process,” it turned out though, and so we got web browsers. Editors could wait.

Thus the web was built, cuts here and subtractions there, until it was something everyone could understand. And then it was everywhere.

Tracing back the tracks.

TrackBack in Movable Type, Pingback in WordPress, and Reblogging in Tumblr

eBay. Hotmail. Geocities. The .com boom. Yahoo!

A decade after the web sputtered to life on a NeXT computer in a nondescript CERN office, it was hard to imagine life without the web. And tech had time to catch up to the vision.

Browsers never did become editors. Personal websites then blogs, though, almost fulfilled Berners-Lee’s vision where “sharing what you knew or thought should be as easy as learning what someone else knew.”

And they brought back bi-directional links for a brief moment in time.

It started as TrackBacks, a tiny feature added to Movable Type in 2002. If you linked to a site in your blog post, Movable Type would send a comment to the site you linked. It established “a connection between authors,” said Movable Type co-founder Mena Trott. You could see who linked to your writing, discover like-minded writers, and see the connections between your favorite blogs.

WordPress tweaked the idea with Pingbacks to prevent spam. Tumblr took it further with reblogging, where you could share a full post and add commentary, resurrecting Xanadu’s concept of embedded quotes.

Xanadu never was fully built; “Fonts and glitz, rather than content connective structure, prevail,” mourned Nelson. But blogs, at least, started building the two-way links between ideas.

The editable web.

Editing Wikipedia and seeing linked pages

Then came the in-browser editor with the Wiki, dreamed up by Ward Cunningham five years after Tim Berners-Lee’s WorldWideWeb, built to explore the way “programming ideas are carried by people as they move between projects.”

Connections, again, were the focus. In Cunningham’s WikiWikiWeb, you could edit any page, which meant anyone could contribute. And to explore how ideas were carried, you could link to anything else in the wiki with CamelCase words. It made editing easy, linking even easier.

Wikipedia, launched six years later as an experiment to let the world build an encyclopedia together, ran with the same idea. Only, CamelCase words got confusing, so two weeks into the project, developer Clifford Adams switched to using double brackets like [[this]] to link pages.

The new link style prevented accidental links to, say, three-letter agencies, and as a benefit let linked text look like standard English, not geek-speak. “Now Wikipedia looked more like a proper work, and not one littered with strange CamelCase”, remarked Andrew Lih in The Wikipedia Revolution.

And so the foundation was laid. Anyone could edit and link anything, and they did, writing billions of words and linking concepts together. Even bi-directional linking came along, with Wikipedia’s somewhat-hidden What links here tool on every page listing the pages that reference it.

Roaming through ideas.

Roam Research with embedded document sections and a document graph

Google almost built two-way links back into the web with the now-depreciated link: query that showed sites that reference a link—something that stayed around in Google Scholar to show research that cites another. But Wikipedia and the few remaining blogs with trackbacks aside, bi-directional links never became a core part of the web.

Thus it fell to the next generation of developers to find better ways to link ideas. “We need better way of organizing the questions we are trying to answer, and making explicit the lines of reasoning we hold,” tweeted developer Connor White-Sullivan in early 2019. Months later, his now-viral app Roam Research would be one of the closest modern cousins to Enquire, Xanadu, and Bush’s Memex.

Roam Research in action

It starts, as you might expect, with bi-directional links, built wiki-style with [[double brackets]], building on Wikipedia’s earlier innovation. That adds a link to the bottom of the linked note in a Linked References section, like Enquire.

That’s enough to connect ideas, to link a mention of an author, say, in multiple notes, and see them all together in the page Roam auto-generated when you linked the author’s name. But what if you want to use a quote from that author in a new article?

That’s where Roam’s block embeds come in. You can search for any section from any note and embed it into a new document, Xanadu style. Link ideas, build new connections, then create mashup documents that combine the best of both. Since it’s your private notes, there’s no fear of spam and clutter as with Trackbacks, nor fear of others ripping off your work via mashups. Somehow, finally, the pieces all came together.

The dream was there all along, envisioned by Vannevar Bush as projected articles on a wall, linked together through research. But ideas are hard to sell, sometimes. Maybe we needed a couple decades with links and browsers before editing, bi-directional links, and embedding made sense. It took wiki’s simpler links, blogger John Gruber’s Markdown for more readable copy, Trackbacks’ connected posts, YouTube-style embeds, and time for it all to come together.

Roam’s not alone, either. Wikis offered much of the same idea for the past decades, albeit cluttered with extra markup and features. Notion took the web’s existing embeds to the next level, making it easier to pull work and inspiration from a variety of apps into one document in a more visual Xanadu.

But Roam, with its sparse layout and focus on the core features of bi-directional linking and embedding, finally pulled the core elements together into a tool that could link ideas.

Perhaps Ted Nelson was right about fonts and frills. Maybe plain linked ideas were what we needed all along.

Enquire within upon everything.

macOS dictionary turns every word into a link.

Though perhaps, even bi-directional links and embedded sections may not be enough. You still have to remember to link things, decide in advance what may be of interest later, and make the mental connections to embed ideas from other documents.

What if instead, the computer did that for you, turning everything into links and letting you connect things that in the past you never knew you’d find interesting?

It’s almost an easter egg in macOS’ Dictionary app. Open a definition, then hover over the text. And like magic, every word is a link. Click any word to jump to its definition, building a trail of meaning as you discover new words. It makes Wikipedia better; even if no one thought to link something, the Dictionary app makes the connection for you.

Now imagine that for your notes, your research, your team’s work. You could actually enquire upon everything, no foresight required.

“Computers might not find the solutions to our problems,” wrote Tim Berners-Lee, “but they would be able to do the bulk of the legwork required, assisting our human minds in intuitively finding ways through the maze.”

They just might need more links.

Join the discussion.

Love Roam Research, or still prefer Notion? Join the debate:

Image Credits: Header photo by @hyeryi via Unsplash. WorldWideWeb screenshot via CERN’s 2019 WorldWideWeb Rebuild. TrackBack screenshots via MovableType documentation. Wikipedia visualization by Chris Harrison

NormanChella's avatar
@NormanChella (replying to maguay )
3 years ago

Love this. Always curious about what's coming up after this. Will Roam evolve to add in more features to link beyond blocks? Are there going to be additional features that help with linking collected notes and data together? It's always exciting to read all of this up.

These are just tools to help give you guidance in thinking better after all - they prompt you with linkages, and you create the value. It immediately clicked for me when blocks could be referenced to the first time when I had my own dB, and I'm eager to see where this is going. Thank you for the write up!

2 points
maguay's avatar
@maguay (replying to @NormanChella )
3 years ago

Thank you, glad you enjoyed it!

Roam Research has consistently added new features over the past few months, most recently a Pomodoro timer, which is an interesting step towards having essentially mini apps that live inside Roam Research. The bigger thing coming, though, I think will be if/when you can link to public databases.

2 points
What's the best way to get feedback about a design or product?

Do you have a favorite survey or poll tool to get feedback, or a unique process to get actionable insights from your users about your design, product features, pricing, and more? There was an inte...

What's the best tool for building a customized API documentation?

Let's say you have an API you want to document and you want to: - Let your users test their calls to the API thanks to a "playground" like the one on the Figma API - Be able to customize it to matc...

The community for power users.