Schema as Code with Data API Platform Neurelo
Cloud data API company Neurelo has launched with a 1.0 version of its core product: a database abstraction layer to make distributed scaling easier.
This is downplaying what Neurelo can do somewhat, and to understand the value requires understanding the practical reality of how developers work with databases.
“Today, the industry presents only two options,” says Chirag Shah, CEO and co-founder of Neurelo. “One is you’re going to be dependent on database drivers and a query language.” This approach tends to be tightly coupled to the specific database technology you’ve chosen. Developers often aren’t deeply familiar with the database query language. Not the way a database administrator is. Developers tend to use helper tools like query builders that translate their intentions into the actual query code.
“The second option, if you want to uplevel a little bit, you’re going to use an ORM (Object Relational Mapping) or ODM (Object Document Mapper) in MongoDB,” says Shah. These mappers convert between higher level code in the developer’s chosen programming language and the low-level database query language itself. This approach has ease-of-use benefits, but the generated queries are often challenging to debug and tune.
Query builders and ORMs are functional, but can be a bit rough. The way they build queries can put a lot of stress on databases. As systems scale, tuning application performance requires someone to understand both the database and they developer-level abstraction. Getting useful telemetry information can be tricky, too, and then a developer needs the expertise to make use of it.
Neurelo offers an alternate approach: Why not put the query builder, performance monitoring, telemetry and similar data services in the one place? If developers are going to work with an abstraction anyway, why not use one that solves these kinds of problems for them, and more?
“What if all your reads and writes from your software to your database became an API in the cloud?” says Shah. “Developers will not have to write a single line of database-specific code. They just get all the standard REST and Graph APIs.”
“As a developer, if you come to Neurelo, the only thing that matters is your data model,” he adds.
By using an API-based data intermediary, developers can use data in an application-centric way, decoupling themselves from the specifics of database infrastructure. The data model itself, the schema, becomes another kind of infrastructure-as-code that can be managed using the same tools that are being adopted everywhere else in modern infrastructure stacks.
Database specialists can concentrate on maintaining and improving this data access system for everyone. As systems scale, particularly with newer distributed architectures, these kinds of loosely-coupled abstractions become vital. Reworking application code every time the database changes, and vice-versa, takes a lot of effort, often painstakingly manual.
Neurelo also has AI capabilities. Building complex queries can be very challenging, yet the tightly constrained domain of highly-structured database schema lends itself very well to machine-learning approaches. Generating complex queries from natural language prompts can quickly get 80-90% of the way there, according to Shah.
Not every application will want this kind of bump-in-the-wire abstraction due to very tight latency requirements, but these are quite rare. For most applications, performance will improve by letting specialists focus on tuning Neurelo while developers are freed from having to troubleshoot database query logic in their application code.
There are other approaches, such as API gateways, but they tend to be more generic systems that don’t offer the specialist features that DBAs want. Some database systems also offer API interfaces directly, but these tend to be specific to that database technology, recreating the same query language issues that developers already struggle with.
For those working with a lot of data, there’s merit in having a common piece of data access infrastructure like Neurelo to help developers focus on their task and letting data specialists focus on what they’re best at.