IM Engineer Study Notes
← Back to list
Database
#NewSQL#분산SQL#수평확장#분산트랜잭션#강한일관성
Last updated · 2026-10-20

NewSQL

1. Overview

NewSQL refers to a class of next-generation distributed relational databases designed to preserve the strong consistency, ACID transactions, and standard SQL interface of relational databases (RDBMS) while simultaneously achieving the horizontal scalability (scale-out) and high availability demonstrated by [[nosql]]. The term was first coined in 2011 by Matthew Aslett, an analyst at the 451 Group, to describe "emerging databases that scale OLTP while keeping SQL guarantees."

For a long time, traditional data-processing technology forced a choice between two conflicting axes. On one side stood the single-node RDBMS, proven over decades. It offered the powerful weapons of a normalized schema, ACID transaction guarantees, and declarative SQL, but once throughput hit the limits of a single server, the only path was vertical scaling (scale-up) — replacing it with a bigger machine — which ran into walls of cost and ceiling. On the other side stood NoSQL, which appeared in the late 2000s. It easily gained horizontal scalability and high availability through sharding, but at the price of relaxing strong consistency to eventual consistency and largely abandoning transactions, joins, and standard SQL spanning multiple rows and tables.

NewSQL directly challenges this old dichotomy that "to gain scalability you must give up consistency." In other words, the essence of NewSQL is to offer developers ACID and SQL unchanged, while underneath hiding a distributed system that automatically partitions, replicates, and synchronizes data. In an essay answer, NewSQL must not be reduced to merely a "scalable RDBMS"; it should be described along three axes: (1) a distributed architecture that transparently hides sharding and replication from the application, (2) a distributed transaction and consensus mechanism that preserves serializability even in a distributed environment, and (3) the preservation of SQL and ACID that distinguishes it from NoSQL.

A. Background and Necessity

First, the explosive demand for web-scale transactions is the decisive driver. Businesses that process tens of thousands to hundreds of thousands of writes per second — global e-commerce, fintech payments, billing in large-scale games — while demanding that "there must never be a duplicate order or double payment" have grown. NoSQL's eventual consistency struggles to guarantee the correctness of such financial-grade transactions, and a single RDBMS cannot handle the throughput, so a solution combining the strengths of both became urgently needed.

Second, there is the backlash from operating NoSQL. Many enterprises adopted NoSQL for scalability, but suffered from the complexity of implementing joins, aggregations, and transactions directly in the application layer, and from subtle data-consistency bugs induced by eventual consistency. Field feedback of "we miss SQL and transactions" translated into demand for NewSQL.

Third, the maturation of hardware and distributed technology made NewSQL real. As large memory, fast networks, and proven distributed consensus algorithms such as Paxos and Raft ([[distributed-consensus]]) became commonplace, "horizontal scaling that maintains strong consistency" — once only theory — became implementable at commercial grade. Google's 2012 Spanner paper, proving that external consistency could be achieved at planetary scale, became the decisive turning point of this trend.

2. Overall NewSQL Architecture

A NewSQL system looks like a single giant RDBMS from the outside, but internally it has a distributed structure in which many shared-nothing nodes divide the data into fine pieces and share the load. Its core design philosophy is the "separation of the SQL layer and the storage layer." The application throws SQL as it would at an ordinary RDBMS, and beneath it a distributed SQL engine splits the query across multiple nodes, executes it, and gathers the results.

graph TB
    APP["Application(standard SQL·JDBC)"] --> LB["Load balancer·gateway"]
    LB --> SQL1["SQL node(parser·optimizer·distributed exec)"]
    LB --> SQL2["SQL node(parser·optimizer·distributed exec)"]
    SQL1 --> TXN["Distributed transaction coordinator(timestamp·2PC)"]
    SQL2 --> TXN
    TXN --> SH1["Shard/Range 1(Leader + replicas)"]
    TXN --> SH2["Shard/Range 2(Leader + replicas)"]
    TXN --> SH3["Shard/Range 3(Leader + replicas)"]
    SH1 -. "Raft/Paxos replication" .- R1["Replicas A·B"]
    SH2 -. "Raft/Paxos replication" .- R2["Replicas A·B"]
    META["Metadata·placement manager(shard location·rebalancing)"] --> SH1
    META --> SH2
    META --> SH3

In the structure above, the application connects via standard SQL and drivers (JDBC, the PostgreSQL wire protocol, etc.), so code that used an existing RDBMS can be reused almost unchanged. A SQL node parses and optimizes the incoming query, learns which data lives on which shard from the metadata manager, then sends query fragments only to the relevant shards for parallel execution. The storage layer divides data into shards by key range or hash, and each shard is in turn synchronously replicated to several replicas via Raft/Paxos, so that even if one node dies a leader is automatically elected and service is not interrupted. The crux is that all of this partitioning, replication, and failure recovery must operate transparently, invisible to the application — and this is the decisive point where NewSQL differs from the old way of manually implementing [[db-sharding]].

A. Distributed Transactions and the Consensus Mechanism

NewSQL's hardest challenge is preserving ACID even in transactions that modify data scattered across multiple shards. On a single node, local locks and logs suffice, but once data is spread across several nodes, the atomicity of "all commit or all roll back" must be guaranteed across the network. For this, NewSQL commonly uses a two-layer structure that coordinates atomic commits across multiple shards with classic [[two-phase-commit]] (2PC), while handling the replication within each shard via Raft/Paxos consensus, which is more failure-resilient than 2PC.

A subtler problem here is isolation, and serializability in particular. In a distributed environment, the clocks of different nodes drift, making it hard to decide "which transaction came first." Google Spanner introduces TrueTime, a time API that bounds the error margin to within a few ms using GPS and atomic clocks, and by deliberately waiting out the uncertainty interval at commit time (commit-wait) achieves a globally consistent order — that is, external consistency. CockroachDB, by contrast, combines physical time and a logical counter with a Hybrid Logical Clock (HLC) on commodity hardware without special equipment, and guarantees serializability (SERIALIZABLE) through timestamp refresh (read refresh) on reads. Thus "how to handle the distributed clock" becomes the core technical choice that distinguishes each NewSQL product.

B. How Sharding and Automatic Rebalancing Work

NewSQL's horizontal scalability comes from its ability to split data into small pieces and automatically relocate those pieces as nodes are added or removed and load changes. CockroachDB calls this unit a range and TiDB calls it a region; they are typically split into units of a few hundred MB, each responsible for a particular key range. When a piece grows too large it is split, and when load concentrates on a particular node it is moved (rebalanced) to another — the system performs this entire process on its own.

graph LR
    START["Write inflow·data growth"] --> CHK{"Range size/load exceeded?"}
    CHK -->|"No"| KEEP["Keep state·replica sync"]
    CHK -->|"Yes"| SPLIT["Range split(split)"]
    SPLIT --> DECIDE{"Load imbalance across nodes?"}
    DECIDE -->|"Yes"| MOVE["Replica move·rebalance"]
    DECIDE -->|"No"| PLACE["Keep placement"]
    MOVE --> RAFT["Raft group reconfig·new leader election"]
    PLACE --> RAFT
    KEEP --> RAFT
    RAFT --> SERVE["Continuous uninterrupted service"]
    SERVE --> START

What matters in the flow above is that rebalancing happens online, without service interruption. Even when moving a replica, a new replica is first added to the Raft group and allowed to catch up with the data before the old one is removed, so a quorum (majority) is always maintained and reads and writes are never cut off. For example, when traffic surges on Black Friday, adding a few nodes lets the system automatically distribute the hot ranges to the new nodes, so throughput grows near-linearly within minutes — this is the reality of the elasticity that NewSQL advertises.

3. Core Technical Elements

The technologies NewSQL mobilizes to achieve its goal of "scalability and consistency at the same time" span several layers, each a convergence of distributed-systems research that evolved independently. The table below is a supplementary summary; "why each technology is needed" must be understood in prose.

Layer Core technology Problem solved
Storage·partitioning Range/Hash sharding, auto rebalancing Horizontal distribution of data·load
Replication·availability Raft/Paxos sync replication, auto failover Uninterrupted on node failure([[database-replication-high-availability]])
Transaction Distributed 2PC, MVCC([[mvcc-snapshot-concurrency-control]]) Multi-shard atomicity·concurrency
Time·ordering TrueTime, HLC, timestamp oracle Distributed serializability·external consistency
Query Distributed SQL optimizer, push-down Standard SQL into parallel node execution

In particular, MVCC (multi-version concurrency control) is the hidden foundation of NewSQL performance. By managing multiple versions of each row by timestamp so that reads do not block writes, a short OLTP write can flow without delay even while a long-running analytical query is in progress. Also, the push-down optimization — pushing filter and aggregation operations down to the storage node where the data lives — reduces the volume of data moved over the network, and is a decisive factor governing query performance in a distributed environment.

4. Classifying NewSQL

In a 2016 joint paper, Andy Pavlo and Matthew Aslett classified NewSQL into three branches by implementation approach, and this framework remains a valid basis for description.

First, entirely new architectures (New Architectures). Engines designed from the ground up on the premise of distribution, such as Spanner, CockroachDB, TiDB, VoltDB, and YugabyteDB, with a shared-nothing structure and native distributed transactions. This is the purest NewSQL, strongest in both scalability and consistency, but with high implementation complexity and maturity risk.

Second, transparent sharding middleware (Transparent Sharding Middleware). Approaches such as Vitess (MySQL-based, developed by YouTube) and Citus (a PostgreSQL extension) that place a sharding layer in front of several existing single RDBMS instances so that the application sees them as one DB. By reusing proven RDBMS engines they are stable, but the strength of their distributed transactions is not as high as that of the new architectures.

Third, DBaaS (Database-as-a-Service). Forms such as Amazon Aurora and Google AlloyDB, in which a cloud provider separates storage and compute and offers them as a managed service, shifting the operational burden to the provider while enjoying automatic scaling.

5. Comparing RDBMS·NoSQL·NewSQL

The difference among the three families must be understood not as a simple feature list but as a difference in design philosophy of what each tried to preserve by sacrificing what.

Category Traditional RDBMS NoSQL NewSQL
Scaling method Vertical(scale-up) Horizontal(scale-out) Horizontal(scale-out)
Consistency Strong consistency(ACID) Eventual consistency(BASE) Strong consistency(ACID)
Query Standard SQL·joins Limited·API-centric Standard SQL·joins
Transaction Full support Limited(single key) Distributed transaction support
Representative Oracle, MySQL MongoDB, Cassandra Spanner, CockroachDB, TiDB

If the RDBMS sacrificed scalability to preserve consistency and SQL, NoSQL sacrificed consistency and SQL to gain scalability and availability. NewSQL "preserves both, but pays that cost in distributed consensus and network latency" — that is, it is not free; under the constraints of the CAP theorem it buys consistency by paying in latency and geographic replication cost. Therefore the claim "NewSQL is unconditionally superior" is wrong, and the trade-off of increased write latency in a geographically distributed environment must always be mentioned alongside.

6. Comparison of Implementations

Descending to concrete products, the differences in design choices become distinct. Google Spanner presupposes TrueTime hardware (GPS·atomic clocks) and achieves planetary-scale external consistency, underpinning the trillion-won-scale transactions of Google's ad system (F1) — though it is tied to special infrastructure and is effectively Google Cloud only. CockroachDB was inspired by Spanner but implements serializability with HLC on commodity hardware so it can run anywhere, and raises portability with PostgreSQL wire compatibility. TiDB separates a MySQL-compatible SQL layer from TiKV, a Raft-based distributed KV, and adds the columnar storage engine TiFlash to also encompass [[htap]] — its distinctive point being the processing of both OLTP and real-time analytics on a single copy of data. In Korea too, adoption of TiDB and CockroachDB is increasing in large-scale e-commerce, gaming, and finance, with ongoing reports of operating hundreds of TB of data as a single logical DB while greatly reducing the operational burden of sharding.

7. In Depth: Redefinition and Convergence into "Distributed SQL"

Recently the industry prefers the name "Distributed SQL" over the somewhat vague term 'NewSQL'. This is a narrower, clearer category of "a DB designed from the ground up on the premise of distribution and providing standard SQL and strong consistency," into which CockroachDB, YugabyteDB, TiDB, and Spanner are repositioning themselves. It is an attempt to distinguish from the simple sharding middleware or in-memory OLTP products once included in early NewSQL.

A second trend is storage-compute separation and going serverless. Billing models that automatically scale down to zero and back up with usage, such as CockroachDB Serverless and TiDB Serverless, have appeared, so even small- and medium-scale services can enjoy the benefits of a distributed DB without an initial cost burden. A third is convergence with HTAP and AI workloads. This is the direction of integrating row storage and column storage in one system to jointly provide real-time analytics and vector search on operational data, with the boundaries of transaction, analytics, and AI blurring inside the database. That said, distributed SQL still has larger write latency than a single-node RDBMS and demands much operational expertise, so it remains true that it is not an all-purpose solution for every workload.

8. Considerations and Implications

First, selective application according to the level of consistency required is key. NewSQL suits core OLTP that simultaneously needs strong consistency and scalability, such as global payments and inventory, but forcibly moving work that a single region handles fine onto distributed SQL only increases write latency and operational complexity. One should first verify "does scalability really exceed the limits of a single RDBMS" before deciding to adopt.

Second, the trade-off of latency, consistency, and geographic distribution must be designed quantitatively. Multi-region synchronous replication can increase write latency by tens of ms or more because of network round trips, so the design capability to control latency with a placement policy (per-region partitioning) that exploits data locality decides success or failure.

Third, operational maturity and ecosystem risk must be considered. New-architecture NewSQL is relatively young, so the tools and skilled personnel for failure handling, backup, and monitoring are not as abundant as for a traditional RDBMS. Through a PoC one must verify failure-injection (chaos) scenarios and upgrade/migration paths, and also check vendor lock-in and the possibility of open-source license changes.

Fourth, a transition and coexistence strategy with existing assets is needed. Rather than wholesale replacement, it is realistic to migrate gradually from the scaling-bottleneck workloads via the Strangler approach, and to design uninterrupted transition with data migration and CDC-based synchronization. Since standard SQL and PostgreSQL compatibility greatly lower porting cost, the maturity of compatible protocols and migration tools should be a core evaluation item when selecting a product.

Fifth, from the professional-engineer perspective of outlook, NewSQL/distributed SQL is establishing itself as the default data layer of cloud-native, multi-region services, and is expected to evolve into an integrated data platform that processes "transactions, analytics, and intelligence on a single copy of data" by absorbing HTAP, vector, and AI capabilities.

References

In one line: NewSQL (distributed SQL) is a distributed relational DB that, through sharding, Raft replication, distributed transactions, and TrueTime/HLC-based distributed clocks, preserves the ACID and standard SQL of the RDBMS while achieving the horizontal scalability of NoSQL; the key is to understand the trade-offs of latency, consistency, and operational complexity and apply it selectively to core OLTP.