Show HN: Stargate – An open source API framework for data
By The Stargate Authors
Sep 14, 2020
We’re enraged to delivery the first preview of Stargate, an delivery source API framework for info. This venture got right here about because of we purchased drained of the utilize of assorted databases and varied APIs looking out on the work that we had been trying to receive done. With “read the manual” fatigue and lengthy choice processes wearing on us each time we created a brand contemporary venture, we thought – why no longer originate a framework that can support many APIs for a spread of workloads? From this, Stargate became born.
To this level we’ve started with Apache Cassandra as the first backend and implemented the Cassandra Quiz Language (CQL) and a REST API for CRUD receive correct of entry to to info in tables and dangle many more APIs coming soon.
For any venture to thrive it desires to be straight forward to work with and make contributions to. We kept that in thoughts while architecting Stargate and the core codebase is modularized to facilitate extensibility and collaboration. We hope that you’ll join us on this race to redefine what’s doubtless in the case of interacting with info at scale and as soon as you happen to’re attracted to getting concerned you would possibly maybe examine out the code on GitHub, join our mailing listing, or note us on Twitter @stargateio! 🚀
How does Stargate work?
Stargate is an info gateway element that is deployed between your client applications and database. We chose Cassandra as the first database because of it solves the world’s hardest scale and availability challenges and we deem there’s low striking fruit in the case of APIs for the database.
Stargate itself is in keeping with the theorem that of a Cassandra coordinator node and is amazingly equivalent to the “elephantine client” that Eric Lubow explains in his presentation at Cassandra Summit in 2016. This implies when Stargate is deployed, it joins the Cassandra cluster as a coordinator node but does not retailer any info. We chose this make because of coordinator nodes in Cassandra already care for many of the question coping with and routing that’s the major for a extremely accessible storage proxy and it made sense to reuse that time-tested common sense. This structure lets in for compute to be scaled independently of storage; a protracted-established mannequin when the utilize of cloud infrastructure.
The high-stage structure plan below explains the put Stargate matches in the stack and the vision for contemporary APIs and integrations.
Taking a closer scrutinize, when a question is dispensed to Stargate, it’s handled by the API Service, translated to the database query, and dispatched to the Persistence Service. The Persistence provider then sends the question to the storage replicas of that row the utilize of Cassandra’s internal QueryHandler. The Persistence Service processes the question and responds to the customer as soon because it receives acknowledgements from the sequence of storage replicas specified by the question consistency stage.
Here’s a reasonably traditional implementation of “Dynamo-based totally mostly” coordination. Stargate’s closing diagram is to dangle pluggable APIs on the front-reside and pluggable storage engines on the befriend-reside alongside with your complete Dynamo magic taking place in the heart.
To be conscious how this dispensed coordination works noteworthy more, in a single 6-node datacenter cluster with 3 storage replicas, an incoming question will scurry to all 3 replicas that dangle the requested row. On this case, the nodes labeled 1, 3, and 4 are storages nodes that dangle the ideas for the question. Within the instance below, the consistency stage is LOCAL_QUORUM meaning that Stargate will read or write info to 2 of the 3 storage replicas in bellow to meet the customer question and ack the success.
Below is an instance of the REST API for CRUD operations that serves the ideas kept in Cassandra tables over HTTP. That you too can build that instance by downloading Postman and importing the Stargate REST API sequence in the documentation.
Step 1: Pull down the docker image (https://hub.docker.com/u/stargateio)
Step 2: Initiate the docker container the utilize of the
DEVELOPER_MODE=factual env variable. This eliminates the must set up a separate Cassandra occasion.
Step 3: Generate an auth token
Step 4: Get a keyspace
Step 5: Get a table
Step 6: Add rows
Step 7: Get rows
Voila! Yow will stumble on the rotund reference for the table-based totally mostly REST API and more in the clinical doctors.
How can I change into enthusiastic?
Take a look at out the code and launch constructing with us! We’ve architected this venture with exploration, flexibility, and API and storage agnosticism as our first class electorate; the diagram is that this makes it straight forward for the community to innovate and add contemporary APIs and extensions.
The plan below reveals how the Stargate modules match collectively.
To receive started constructing a brand contemporary API extension, dangle a search at the reward REST API implementation in the Stargate repo first. To receive started constructing a brand contemporary storage extension, examine out the persistence-cassandra-3.11 implementation. After you dangle an idea of what you cherish to dangle to originate, shoot us a gift on the mailing listing to focus on. Show that the internal APIs are below filled with life pattern and so they’re inclined to alternate earlier than the first version is released.
To query contemporary elements or file bugs, originate a brand contemporary GitHub self-discipline in the stargate/stargate repository and we’ll eradicate it from there.
Let’s Come at some level of
We can’t wait to scrutinize the put this venture takes us and we’ll be rolling out a pair of contemporary APIs in the coming weeks. That you too can receive updates by following us on Twitter @stargateio and throw us a smartly-known particular person on Github as soon as you happen to cherish what you peek!
We’re ready for steal-off, peek you in the celebrities! ✨