Show HN: Stargate – An open source API framework for data

0

The Stargate Authors By The Stargate Authors

Sep 14, 2020

We’re enraged to delivery the first preview of Stargate, an delivery source API framework for info. This venture got right here about because of we purchased drained of the utilize of assorted databases and varied APIs looking out on the work that we had been trying to receive done. With “read the manual” fatigue and lengthy choice processes wearing on us each time we created a brand contemporary venture, we thought – why no longer originate a framework that can support many APIs for a spread of workloads? From this, Stargate became born.

For instance, as soon as you happen to’re a JavaScript developer constructing a internet situation, saving and skimming JSON is sizable and maybe you additionally dangle some GraphQL sprinkled at some level of. Whereas you happen to’re a Java or C# developer engaged on an enterprise utility, SQL is seemingly more your model. The diagram of Stargate is to create your info accessible for you by without reference to API you would possibly maybe dream up without reference to the backing datastore.

To this level we’ve started with Apache Cassandra as the first backend and implemented the Cassandra Quiz Language (CQL) and a REST API for CRUD receive correct of entry to to info in tables and dangle many more APIs coming soon.

For any venture to thrive it desires to be straight forward to work with and make contributions to. We kept that in thoughts while architecting Stargate and the core codebase is modularized to facilitate extensibility and collaboration. We hope that you’ll join us on this race to redefine what’s doubtless in the case of interacting with info at scale and as soon as you happen to’re attracted to getting concerned you would possibly maybe examine out the code on GitHub, join our mailing listing, or note us on Twitter @stargateio! 🚀

How does Stargate work?

Stargate is an info gateway element that is deployed between your client applications and database. We chose Cassandra as the first database because of it solves the world’s hardest scale and availability challenges and we deem there’s low striking fruit in the case of APIs for the database.

Stargate itself is in keeping with the theorem that of a Cassandra coordinator node and is amazingly equivalent to the “elephantine client” that Eric Lubow explains in his presentation at Cassandra Summit in 2016. This implies when Stargate is deployed, it joins the Cassandra cluster as a coordinator node but does not retailer any info. We chose this make because of coordinator nodes in Cassandra already care for many of the question coping with and routing that’s the major for a extremely accessible storage proxy and it made sense to reuse that time-tested common sense. This structure lets in for compute to be scaled independently of storage; a protracted-established mannequin when the utilize of cloud infrastructure.

The high-stage structure plan below explains the put Stargate matches in the stack and the vision for contemporary APIs and integrations.

Taking a closer scrutinize, when a question is dispensed to Stargate, it’s handled by the API Service, translated to the database query, and dispatched to the Persistence Service. The Persistence provider then sends the question to the storage replicas of that row the utilize of Cassandra’s internal QueryHandler. The Persistence Service processes the question and responds to the customer as soon because it receives acknowledgements from the sequence of storage replicas specified by the question consistency stage.

Here’s a reasonably traditional implementation of “Dynamo-based totally mostly” coordination. Stargate’s closing diagram is to dangle pluggable APIs on the front-reside and pluggable storage engines on the befriend-reside alongside with your complete Dynamo magic taking place in the heart.

To be conscious how this dispensed coordination works noteworthy more, in a single 6-node datacenter cluster with 3 storage replicas, an incoming question will scurry to all 3 replicas that dangle the requested row. On this case, the nodes labeled 1, 3, and 4 are storages nodes that dangle the ideas for the question. Within the instance below, the consistency stage is LOCAL_QUORUM meaning that Stargate will read or write info to 2 of the 3 storage replicas in bellow to meet the customer question and ack the success.

Get Started

Below is an instance of the REST API for CRUD operations that serves the ideas kept in Cassandra tables over HTTP. That you too can build that instance by downloading Postman and importing the Stargate REST API sequence in the documentation.

Step 1: Pull down the docker image (https://hub.docker.com/u/stargateio)

docker pull stargateio/stargate-3_11:v0.0.3

Step 2: Initiate the docker container the utilize of the DEVELOPER_MODE=factual env variable. This eliminates the must set up a separate Cassandra occasion.

  docker lumber --name stargate -p 8081: 8081 -p 8082: 8082 -p 127.0.0.1: 9042: 9042 -d -e CLUSTER_NAME=stargate -e CLUSTER_VERSION=3.11 -e DEVELOPER_MODE=factual stargateio/stargate-3_11:v0.0.3

Step 3: Generate an auth token

  curl -L -X POST 'http://localhost: 8081/v1/auth' -H 'Order-Kind: utility/json' --info-raw '{
    "username": "cassandra",
    "password": "cassandra"
  }'

Expected Output:

  {"authToken": "{auth-token-right here}"}

Step 4: Get a keyspace

  curl --space --question POST 'localhost: 8082/schemas/keyspaces' 
    --header 'X-Cassandra-Token: {auth-token-right here}' 
    --header 'Order-Kind: utility/json' 
    --info-raw '{
      "name": "users_keyspace",
      "replicas": 1
    }'

Expected Output:

  {"name": "users_keyspace"}

Step 5: Get a table

  curl --space --question POST 'localhost: 8082/schemas/keyspaces/users_keyspace/tables' 
    --header 'X-Cassandra-Token: {auth-token-right here}' 
    --header 'Order-Kind: utility/json' 
    --info-raw '{
      "name": "users",
      "columnDefinitions": [
        {
          "name": "firstname",
          "typeDefinition": "text"
        },
        {
          "name": "lastname",
          "typeDefinition": "text"
        },
        {
          "name": "email",
          "typeDefinition": "text"
        },
        {
          "name": "favorite color",
          "typeDefinition": "text"
        }
      ],
      "primaryKey": {
        "partitionKey": ["firstname"],
        "clusteringKey": ["lastname"]
      },
      "tableOptions": {
        "defaultTimeToLive": 0,
        "clusteringExpression": [
          {
            "column": "lastname",
            "order": "ASC"
          }
        ]
      }
    }'

Expected Output:

  {"name": "users"}

Step 6: Add rows

  curl --space --question POST 'localhost: 8082/v2/keyspaces/users_keyspace/users' 
  --header 'X-Cassandra-Token: {auth-token-right here}' 
  --header 'Order-Kind: utility/json' 
  --info-raw '{
    "firstname": "Mookie",
    "lastname": "Betts",
    "electronic mail": "mookie.betts@gmail.com",
    "favourite coloration": "blue"
  }'

Expected Output:

  {"firstname": "Mookie","lastname": "Betts"}

Step 7: Get rows

  curl -G --space 'http://localhost: 8082/v2/keyspaces/users_keyspace/users' 
    --header 'X-Cassandra-Token: {auth-token-right here}' 
    --header 'Order-Kind: utility/json' 
    --info-urlencode 'the put={"firstname": {"$eq": "Mookie"}}'

Expected Output:

  {"count": 1,"info": [{"firstname":"Mookie","favorite color":"blue","email":"mookie.betts@gmail.com","lastname":"Betts"}]}

Voila! Yow will stumble on the rotund reference for the table-based totally mostly REST API and more in the clinical doctors.

How can I change into enthusiastic?

Take a look at out the code and launch constructing with us! We’ve architected this venture with exploration, flexibility, and API and storage agnosticism as our first class electorate; the diagram is that this makes it straight forward for the community to innovate and add contemporary APIs and extensions.

The plan below reveals how the Stargate modules match collectively.

To receive started constructing a brand contemporary API extension, dangle a search at the reward REST API implementation in the Stargate repo first. To receive started constructing a brand contemporary storage extension, examine out the persistence-cassandra-3.11 implementation. After you dangle an idea of what you cherish to dangle to originate, shoot us a gift on the mailing listing to focus on. Show that the internal APIs are below filled with life pattern and so they’re inclined to alternate earlier than the first version is released.

To query contemporary elements or file bugs, originate a brand contemporary GitHub self-discipline in the stargate/stargate repository and we’ll eradicate it from there.
Let’s Come at some level of
We can’t wait to scrutinize the put this venture takes us and we’ll be rolling out a pair of contemporary APIs in the coming weeks. That you too can receive updates by following us on Twitter @stargateio and throw us a smartly-known particular person on Github as soon as you happen to cherish what you peek!

We’re ready for steal-off, peek you in the celebrities! ✨

Read More

Leave A Reply

Your email address will not be published.