What is Processeer solving?

Processeer aims at leveraging 2 things: the burden of creating custom reports, and the difficulty of sharing them with other people. First, it simplifies the process of writing scripts that interpret data from APIs, and provides tools to show that data in a beautiful fashion. Second, it makes it easy to share both the results of such scripts and the scripts themselves with other people, encouraging them to reuse and improve existing reports.

Here are few comparison points that might help you understand what it is. Do you know about re:dash? It's kind of the same thing, except it doesn't talk to a database but to APIs, and there's some powerful composition possibilities in Processeer. Maybe you've heard of bl.ocks.org? Processeer wants to give you the same kind of easy-sharing reports.

How about a simple example? Let's say you're a Lord of the Rings fan and you want to know at a glance the repartition of each of Middle Earth's peoples in the community of the Ring as well as the number of Rings of Power each of those peoples has. Well, here's a nice line chart that shows just that.

Right, that's cheating: that last one doesn't even pull data from an API. Alright, how about something a bit more complex? This one tells you if a bug from bugzilla is still relevant or not, by showing the number of crash reports associated to that bug in the last week: Bug Signature Status. That Report is composed of 3 different Blocks. The first one pulls a list of signatures from crash-stats, the second one pulls the bug title from bugzilla, and the third one pulls from crash-stats the actual number of crash reports for each signature we received from the first Block. If you want to look at how it works, simply log in with a github account and you'll be able to edit the Report and Blocks.

Note that you can type in any bug number in the "bug" filter at the top, and click "Run" to rerun that Report for that bug.

How does it work?

Processeer has a few concepts that need to be explained. First of all, you need to have a data source that is accessible over HTTP and that is serving JSON. That includes, for example, the crash-stats public API, bugzilla's REST API, github's REST API, and many more.

Processeer's high level component is called the Report. It is an ordered list of Blocks (we'll learn about that in just a few moments). When you load a Report page, Processeer is going to run each Block in turn, passing the output of the first Block as the input of the second, and so on. The output of the last Block of the chain is then going to be parsed and displayed as nicely as possible, as a table or as a chart for example.

Blocks are the building elements of Processeer. It's where you put most of the logic: where to pull the data from, what to do with it, and what to return. The Models of a Block define how the data is pulled. You can set several Models in a Block. Each one is basically a construct to make an HTTP call to some API that returns JSON. The Controller is a JavaScript function that you write in order to transform the data you get from your models into what you want your Block to return. It can be a specially parsed object that Processeer will parse and turn into a nice looking chart or table, or it can be an object that should be passed to the next Block in a Report. And that is where Params come in. Params define the "input" of your Block: you can set default values for some arguments you expect, and if those arguments exist in the output of the previous Block, their values will be passed down for you to use in your current Block. That mechanism allows, for example, to have a block that pulls some data from an API, then have another Block that uses that very data in the query of another API request. It also makes Blocks more reusable, and encourages users to keep them as small and focused as possible.

Schema of how Processeer works, a recap of the previous paragraph.

How do I use it?

Go to processeer.io and log in using a github account. You will then be able to create new Blocks and Reports. There are some Blocks already, I invite you to take a look at them to see how the system works, and what you can do. There's some documentation on the Processeer github repository, as well as some basic examples to understand the possible outputs.

I am very happy to answer questions if you have any. You can reach out to me with github issues, by email via my contact page, or on IRC (for example in #processeer in Freenode's IRC server).

The future is unwritten

The future of Processeer is unclear. I don't have any users just yet, probably for a lack of advertisement and documentation. But it is hard to invest time into those things when you don't have any users. So, here's my message in a bottle that I am throwing into the sea. If you find Processeer useful and think it can help you, I would love to hear about it. I need fearless explorers ready to dig into it to make the best out of the amazing possibilities it offers.

And if you think it's a brilliant idea and want to contribute, it's all open source (not licensed yet though). :)