GForge Next – The API

A couple of weeks ago, Tony shared some screen shots of GForge Next, and talked about how we’re designing it to Get Out of Your Way. Today, I’d like to talk about the other end of GForge Next – the API side.

We’ve Always Had An API

It’s true – the current GForge has an API, and it has for over ten years. But it’s……not awesome.

If you have ever worked with SOAP services, then you know they can be overly complex, verbose and yet inscrutable. Not to mention it can be fragile, especially between different platforms (I’m looking at you, Java/Axis/Axis2/XFire).

But the real flaw in our SOAP API is that is was bolted on to the existing code. There’s quite a bit of logic shared between the GForge UI and the SOAP API, but also some duplicate code, so the API won’t always act the same way as the good old web UI. As we continue to fix bugs and add features, it continues to be an extra effort to maintain these separate code modules and keep them working the same way.

Now With 100% More REST

Like everything else with GForge, we are our own worst critics, and our own first users. So it was easy for us to decide that to be useful, the API had to be an integral part of the system. The entire GForge Next UI runs against the API. Logging in? API call. Getting your list of assigned tasks? API call. When we use GForge Next UI to accomplish tasks, we’re also automatically making sure that integration via the API will work for all of those use cases.

For wire protocol, we chose REST (like pretty much everyone else these days). JSON + HTTP (GET, POST, PUT, DELETE) is a pretty simple, readable and capable combination, and it aligns nicely with the other parts of our technology stack (like AngularJS and Bootstrap).

The API is even self-documenting thanks to code-comment blocks in the underlying services and our /apidoc endpoint. The API docs include all the entities, methods, parameters and even example input and output. As we continue to add and change the API , the docs are automatically up to date.

For the most part, we submit data to the API using the same JS objects we receive from it. That way, we can take an object from a list of values, assign it as a property on another object and submit it. Super-simple on the client side, and pretty easy on the server as well. But for external integration, it’s not so great. Requiring callers to fetch a bunch of lists and sub-objects in order to properly craft a request makes the API less usable and more prone to invalid requests. So, for most requests, we will accept any of the following values for sub-objects (like a project, a status, the user, etc.):

The “official” JS object for that type (retrieved from the API)
The ID or alias for the object, as a number or string value. For example, ID 2000 or the short name mtutty for a user.
A JS object with an ID property, with the same value as the above bullet.

In general, the API back-end will work pretty hard to figure out what value you’re trying to use, and go find it based on the ID or key value submitted. We also support vanilla FORM POST semantics, so that regular HTML forms, curl and wget requests are easy to do.

Complex Can Still Be Simple

So it’s pretty easy to create a basic CRUD API for a set of related tables. The current GForge Next database has about 137 tables, which is a bit more complicated. Add to that the fact that most realistic use cases (including our own UI views) are going to need multiple types of data, and using the API just went back to difficult and confusing. Well, maybe not.

Largely for our own needs, we started adding related data as optional for many API calls. By adding “rel=xyz” to a GET request, you can ask the GForge Next API for additional data that is related to the entity (or entities) you’re requesting. So for example, GET /api/user/mtutty/details?rel=projects,activitylog includes the most-used projects and most-recent activity records for me. That’s all the data needed for the user profile view, in one call.

Don’t need all that extra stuff? Don’t include the rel, and you can get the payload down from 3.6K to 963 bytes.

How About Some Examples?

Here are a couple of ways we’re already using the API (outside of our own UI) to keep project data and knowledge moving, even when things are happening outside of GForge.

Build and Deployment Activity – We use Jenkins for CI/CD (and deploy GForge Next several times every day). Whenever builds start or fail, or whenever a deployments starts or ends, we use the REST API to log an activity record. This is visible in the Project Home page activity view and in the project chat room (which is where we see it most of the time). Now as we’re working on tasks, we can see not only what tasks other people are working on, and their funny cat pictures, we can also see our build kicking off, and code going live, in real time.

Customer Feedback – First Dozen users will notice the green “Feedback” tab on every page in Next. That tab submits data to a separate Tracker called, well, “Customer Feedback”, using the API. Technically it’s part of our site but we have made a conscious effort to keep it separate from the AngularJS application that contains the rest of GForge Next. The Tracker template will be available in GForge Next, and eventually we’ll make the widget available too, so customers can collect feedback from users directly into their GForge Next project.

Where To Go From Here

While we are excited for the new REST API in GForgeNEXT what really excites us is the creative ways developers will use the API. Because the API is new and will continue to develop, we really hope you consider signing up for early access to GForgeNEXT. Those requesting early access will can expect the beta to begin officially over the next couple of weeks which will include immediate access to the API.