Database Topics
45 topics
- Database
Data Virtualization
An integration technology that unifies dispersed heterogeneous data in real time into a virtual layer at query time without replication or movement, presenting a single view — connection·virtual-view·query-optimization (pushdown)·governance layers, comparison with ETL/DW and hybrid design, and ties to the logical data warehouse and data fabric.
- Database
Public Database Standardization Management (Apr. 2023)
Preventive quality-management activities by build stage in the public DB standardization management manual, and its four diagnostic areas and nine diagnostic items.
- Database
Multidimensional Index Structures
The concepts, types, and applications of index structures for multidimensional and spatial data search (R-Tree, KD-Tree, Quad-Tree, Grid File).
- Database
Columnar Storage
An analytics-optimal (OLAP) storage model that physically lays out a table by column to realize selective column reads, high compression, and vectorized execution — covering the structural comparison with row-oriented storage (NSM), dictionary/RLE/delta encoding with data skipping and late materialization, Parquet/ORC hybrid formats and the lakehouse, and professional-engineer considerations such as HTAP, compression trade-offs, and avoiding open-format lock-in, in an essay style.
- Database
Static SQL vs. Dynamic SQL
A comparison of static SQL, fixed at compile time, and dynamic SQL, generated at runtime, in terms of performance, flexibility, and security (injection).
- Database
CRUD Matrix
An analysis tool that expresses the create, read, update, and delete relationships between processes and entities as a matrix to verify data-function consistency.
- Database
Data Quality Management
Data quality management architecture and maturity, quality criteria for structured/unstructured data, and strategies for standardization, prevention, measurement, and accountability.
- Database
Eventual Consistency and Consistency Models in Distributed Systems
An essay-style treatment of the principle of eventual consistency, in which replicas converge once updates stop and replication propagation completes; comparison with strong, causal, and session consistency; versions, logical clocks, conflict resolution, and idempotent retries; CQRS, CRDT, and global service cases; and the design of guarantee levels, observability, and recovery from a Professional Engineer's perspective.
- Database
Database Concurrency Control
Concurrency control techniques (locking 2PL, timestamp, optimistic, MVCC) that prevent anomalies in concurrent transactions, along with isolation levels and deadlock management.
- Database
Graph Databases and Property Graph/RDF Models
An explanation of the structure and modeling procedure of graph databases, which make nodes, relationships, and properties the core of their storage structure to query paths, patterns, and connectivity, along with the selection criteria between property graphs and RDF, applications in querying, analytics, fraud detection, and recommendation, and the boundaries of quality, security, and ledgers.
- Database
Database Normalization
Normalization to eliminate redundancy and prevent anomalies (insertion, deletion, update) — how anomalies arise, lossless decomposition based on functional dependencies, the stages from 1NF to BCNF, and the trade-off with denormalization.
- Database
Limits of the CAP Theorem and PACELC
CAP only explains the C-vs-A choice during a partition. PACELC adds the latency (L) vs. consistency (C) trade-off under normal operation to guide distributed DB selection.
- Database
Open Table Format (Apache Iceberg)
An in-depth treatment of Apache Iceberg, an engine-neutral open table format that gives snapshot-based ACID, hidden partitioning, schema/partition evolution, and time travel to a collection of files on object storage — covering its metadata-layer structure, operating principles, CoW/MoR and compaction, comparison with Delta/Hudi/Hive, and standardization trends.
- Database
Data Governance
A control framework that integrally manages data standards, quality, metadata, security, and life cycle through policy, organization, process, and technology. Data stewardship and standardization are central, evolving toward data mesh, DataOps, and AI governance.
- Database
Database Connection Pool Design and Tuning
Organizes the principles of a connection pool that reuses database connections to control connection-establishment cost and concurrency surges — covering lease/return/validation/timeout, pool sizing, comparison of application pools and external poolers, and failover, observability, and security design.
- Database
Write-Ahead Logging (WAL) and Database Recovery (ARIES)
Explains the WAL protocol of 'log before data' that guarantees atomicity and durability, along with log structure (LSN, physical/logical logging), checkpoints, and the three ARIES phases of analysis, redo, and undo with DPT, CLR, and repeating history via LSN-based scenarios, and organizes the steal/no-force policy trade-off and extension to replication, CDC, and cloud DBs.
- Database
Bitmap Index and Analytical Database Optimization
An index structure that, in low-cardinality, read-centric environments, represents the row set per value as a bit string and accelerates multi-condition analytical queries with AND/OR operations, along with comparison with B-Tree and hash, bitmap joins, DML trade-offs, and data warehouse application strategy.
- Database
The Necessity and Expected Effects of Data Standardization
A foundation of data governance that unifies terms, domains, and codes to consistent criteria to secure consistency, interoperability, and efficiency.
- Database
Data Integration and Migration — Integrity and Consistency
The difference between integrity and consistency in data migration, profiling-analysis perspectives, multi-layered verification methods, and rehearsal, rollback, and CDC cutover strategies.
- Database
Distributed Lock
The structure and protocols of a distributed lock that implements mutual exclusion via atomic acquisition, leasing, owner tokens, and fencing tokens so that multiple servers do not modify a shared resource simultaneously. It compares database, Redis, and consensus-based coordinators, and covers TTL expiration and zombie-client handling, conditional updates, idempotency, and operational strategy.
- Database
Data Storage: Files vs. DB vs. Blockchain
A comparison of files (application-managed), DBMS (centralized ACID), and blockchain (distributed-ledger immutability) in terms of trust authority, integrity, and performance.
- Database
LSM Tree (Log-Structured Merge-Tree)
A write-optimized storage structure that gathers random writes in a MemTable, flushes them sequentially, and organizes immutable SSTables via background compaction — organizing WAL, Bloom filter, tombstone, and MVCC behavior, leveled/size-tiered compaction and the read/write/space amplification (RUM) trade-off, comparison with B+ trees, and RocksDB/Cassandra application cases.
- Database
Two-Phase Commit (2PC) and Three-Phase Commit (3PC) Protocols
2PC (prepare/commit), which guarantees the atomicity of distributed transactions, and 3PC (pre-commit), which mitigates its blocking — covering coordinator/participant structure, log-based recovery, blocking and partition vulnerabilities, and comparison with XA, saga, and Paxos Commit alternatives.
- Database
Data Catalog and Metadata Management
Organizes the concept and structure of a data catalog that links technical, business, operational, and governance metadata to data assets, glossaries, quality, lineage, and policy — covering collection, curation, search, and impact-analysis processes, ISO/IEC 11179 and W3C DCAT alignment, use in data products and AI, and adoption strategy.
- Database
Data Warehouse Modeling Based on Data Vault 2.0
A data warehouse modeling methodology that separates business keys, relationships, and attribute history into Hubs, Links, and Satellites and layers a Raw Vault and Business Vault to secure change resilience, auditability, and parallel loading. It also covers differences from star schema, 3NF, and the lakehouse, along with key, point-in-time, idempotency, quality, and privacy design.
- Database
Public-Sector DB Standardization Guidelines: Table Definition Document
A deliverable that documents table structure, meaning, and constraints in a standard format per the standardization guidelines. Recorded items, authoring guidance, quality diagnosis, and linkage to data governance are covered in depth.
- Database
MVCC (Multi-Version Concurrency Control) and Snapshot-Based Transactions
Organizes MVCC, which reduces read/write contention via per-transaction snapshots and multiple row versions — covering visibility determination, isolation levels, PostgreSQL/InnoDB implementations, version cleanup, locking, serialization retries, and operational design.
- Database
Database Index Structures (B-Tree and B+Tree)
This essay covers the B-Tree, a balanced multi-way search tree that increases fan-out by mapping a node to a disk page, and the B+Tree, which places data only in leaves and links them as a linked list to strengthen range and sort queries — their structures, operations (search, split, merge), and comparison, along with clustered/covering indexes and anti-patterns, and the changes brought by the LSM-Tree and SSD era.
- Database
Database Replication and High-Availability Design
Organizes database high-availability design and operational strategy around synchronous/asynchronous replication, single- and multi-leader topologies, replication lag and read consistency, failover, fencing, RPO, and RTO.
- Database
Database Optimizer (RBO, CBO)
The concept of the optimizer, which selects the optimal execution plan for SQL, a comparison of the rule-based RBO (deprecated) and the cost-based CBO (standard), tuning factors such as statistics, histograms, and hints, and adaptive optimization.
- Database
Database Sharding
A technique that horizontally partitions large volumes of data across multiple DB servers to achieve load balancing and scalability. Covers a comparison with partitioning and the shard key.
- Database
Characteristics of Database Transactions (ACID)
The atomicity, consistency, isolation, and durability (ACID) of transactions and their implementation techniques (locking, MVCC, WAL), isolation levels, and extensions to CAP, BASE, 2PC, and Saga in distributed environments.
- Database
Master Data Management (MDM)
The concept and necessity of MDM, which manages core reference data such as customers and products in a single, unified way, its components, and considerations during implementation.
- Database
Time Series Database
This essay provides an in-depth treatment of the data model of a time series database that treats time as a first-class axis and optimizes append-only high-volume ingestion, time-axis compression (delta/Gorilla), range aggregation, downsampling, and retention policies, along with its LSM-based storage engine, a comparison of product types, and practical application to observability and IoT.
- Database
Database Tuning
The concept and purpose of tuning, which diagnoses and optimizes database performance degradation based on execution plans, and techniques by layer: design (denormalization, indexing, partitioning), DBMS, and SQL (optimizer, hints).
- Database
HTAP (Hybrid Transactional/Analytical Processing)
An architecture that processes OLTP and OLAP in a single engine without ETL through in-memory and row/column hybrid storage and workload isolation, providing real-time analytics on operational data — its implementation types, core technologies, and trade-offs.
- Database
MongoDB
The data model, replica sets, sharding, indexes, and aggregation of MongoDB, a document-oriented NoSQL database that stores data as JSON-like documents (BSON); its comparison with relational databases; and recent trends such as transactions and vector search.
- Database
Data Warehouse and OLAP
This essay covers the structure of a data warehouse, an analytics-only store separated from operational systems (OLTP) that accumulates history under the principles of subject orientation, integration, time-variance, and non-volatility, along with ETL/ELT pipelines, dimensional modeling (star, snowflake, SCD), OLAP operations and a comparison of MOLAP/ROLAP/HOLAP, and its evolution toward the cloud DW and the lakehouse, in a professional-engineer essay format.
- Database
Data Fabric
A technology-centric data-management architecture that weaves discovery, integration, governance, and delivery into a single logical layer across distributed, heterogeneous data—without physically consolidating it—using active metadata, knowledge graphs, and AI automation. It minimizes data movement through data virtualization and automatically enforces lineage-based governance, becoming effective when combined complementarily with the organization-centric data mesh.
- Database
Change Data Capture (CDC)
A data-integration technique that extracts only changes (INSERT, UPDATE, DELETE) in real time from sources such as a source DB's transaction log and propagates them downstream—covering log-, trigger-, and query-based approaches and offset, ordering, idempotency, and outbox patterns.
- Database
Vector Database
A database that performs high-speed similarity search over high-dimensional vectors—embeddings of unstructured data—using ANN indexes. With HNSW, IVF, and PQ indexes plus hybrid search and re-ranking, it serves as a semantic knowledge store for RAG, recommendation, and anomaly detection, where balancing the accuracy–performance–cost trade-off is key.
- Database
RDBMS Data Modeling (Identifying and Non-identifying Relationships)
Conceptual → logical → physical modeling stages and a comparison of identifying relationships (parent PK included in child PK) and non-identifying relationships (ordinary FK), with normalization and integrity considerations.
- Database
The Five Transparencies of Distributed Databases
Location, fragmentation, replication, concurrency, and failure transparency of a distributed DB — concepts that hide distribution, replication, concurrency, and failures so the system appears as a single database.
- Database
NoSQL Types and Data Modeling Procedure
A storage technology optimized for unstructured, large-scale, distributed data — the Key-Value, Document, Column, and Graph types and a query-first denormalized modeling procedure.
- Database
Transaction Isolation Levels
The four isolation levels (Read Uncommitted to Serializable) and the Dirty, Non-repeatable, and Phantom Read anomalies, organized around examples.