CLOUD-NATIVE ARCHITECTURE FOR RETAIL ORDER MANAGEMENT SYSTEMS: DESIGNING AROUND THE AZURE COSMOS DB 2-GB PARTITION LIMIT AND SCHEMA NORMALIZATION
DOI:
https://doi.org/10.65009/kd000269Keywords:
Cloud-native architecture, Retail order management systems, Azure Cosmos DB, Logical partition limit, Schema normalization, Domain-driven design, Event-driven systems, and Scalable data modeling.,,Abstract
Orders management systems (OMS) used in modern retail environments must be capable of
performing a high volume of transactions, possess elastic scalability, and be consistent across
bursty workloads. A combination of a cloud-native design and distributed NoSQL databases
provide a good basis to these needs, but also present non-trivial data modeling limitations. The
2-GB logical partition limit imposed by Azure Cosmos DB is one of the most serious
constraints and can cause partition hot-spots and unrestricted document increase of naively
modelled schemas. The paper provides a new architecture of cloud-native OMS, which designs
systematically within the same constraint by adhering to schema normalized strictly, per-legal
decomposition of data, and domain-oriented segmentation. The data on orders are partitioned
into constrained aggregates in line with the business events and allow controlled partition
growth and predictable scaling behavior. A more event-based interaction model also further
separates order transitions off of persistent storage issues and enhances scale and resiliency.
The proposed architecture illustrates how business domain alignment with appropriate Cosmos
DB partitioning semantics can guarantee the scalability, maintainability, and sustainability of
business operations in insane scale OMS deployments.
References
J. Aguilar-Saborit, “POLARIS: The distributed SQL engine in azure synapse,” Proc.
VLDB Endowment, vol. 13, no. 12, pp. 3204–3216, 2020.
M. Seidemann, N. Glombiewski, M. Körber, and B. Seeger. “Chronicle DB: A High
Performance Event Store. ACM Trans”. Database Syst., 44(4), 2019.
P. Antonopoulos, “Socrates: The new SQL server in the cloud,” in Proc. ACM Int. Conf.
Manage. Data, 2019, pp. 1743–1756.
L. Burkhalter, A. Hithnawi, A. Viand, H. Shafagh, and S. Ratnasamy. “Time Crypt:
Encrypted Data Stream Processing at Scale with Cryptographic Access Control”. In Proc.
th USENIX Symp. on Networked Systems Design & Implementation, pp. 835–850.
, USENIX.
W. Cao, “PolarFS: An ultra-low latency and failure resilient distributed file system for
shared storage cloud database,” Proc. VLDB Endowment, vol. 11, no. 12, pp. 1849–1862,
Y. Afek, A. Bremler-Barr, and L. Shafir, “Network anti-spoofing with SDN data plane,” in
Conf. on Computer Communications, 2017, pp. 1–9.
P. Das, “Amazon SageMaker autopilot: A white box AutoML solution at scale,” in Proc.
th Int. Workshop Data Manage. End-to-End Mach. Learn., 2020, pp. 2:1–2:7.
J. Xing, W. Wu, and A. Chen, “Architecting programmable data plane defenses into the
network with FastFlex,” in ACM Workshop on Hot Topics in Networks, 2019, pp. 161
A. A. Visheratin, A. Struckov, S. Yufa, A. Muratov, D. A. Nasonov, N. Butakov, Y.
Kuznetsov, and M. May. “ Peregreen - modular database for efficient storage of historical
time series in cloud environments”. In Proc. USENIX 2020 Annual Technical Conf., pp.
–601. USENIX.
P. Helland, “Immutability changes everything,” Communications of the ACM, vol. 58,
no. 1, pp. 54–62, 2019.
M Fowler, “Schema evolution patterns for NoSQL databases,” IEEE Software, vol. 36,
no. 2, pp. 68–75, 2018.
J. Lewis and M. Fowler, “Microservices: A definition of this new architectural term,”
IEEE Internet Computing, vol. 22, no. 3, pp. 72–78, 2018.
M. Kleppmann, “Designing Data-Intensive Applications: The Big Ideas Behind
Reliable, Scalable, and Maintainable Systems”, 1st ed., Sebastopol, CA, USA: O’Reilly
Media, 2019.
G. Hohpe and B. Woolf, “Enterprise Integration Patterns: Designing, Building, and
Deploying Messaging Solutions”, Boston, MA, USA: Addison-Wesley, 2018.
J. Humble and D. Farley, Continuous Delivery: Reliable Software Releases through
Build, Test, and Deployment Automation, Boston, MA, USA: Addison-Wesley, 2020.
E. Evans, “Domain-Driven Design: Tackling Complexity in the Heart of Software”,
Boston, MA, USA: Addison-Wesley, 2019.
S. Newman, “Building Microservices: Designing Fine-Grained Systems”, 2nd ed.,
Sebastopol, CA, USA: O’Reilly Media, 2019.
L. Chen, “Microservices: Architecture, containerization, and scalability,” IEEE Cloud
Computing, vol. 5, no. 2, pp. 16–23, 2018.
R. Buyya, S. Narayana Srirama, G. Casale, R. Calheiros, Y. Simmhan, B. Varghese, E.
Gelenbe, B. Javadi, L.Miguel Vaquero, and M. Parashar, “A manifesto for future
generation cloud computing: Research directions for the next decade,” IEEE Cloud
Computing, vol. 5, no. 4, pp. 52–60, 2018.
S. Sakr and A. Liu, “Big data processing in cloud environments,” IEEE Transactions
on Services Computing, vol. 13, no. 2, pp. 227–240, 2020.
M. Kleppmann and N. Chernyak, “Partitioning and replication strategies for scalable
distributed databases,” IEEE Data Engineering Bulletin, vol. 42, no. 1, pp. 5–16, 2019.
H. V. Jagadish, Johannes Gehrke, Alexandros Labrinidis, Yannis Papakonstantinou,
Jignesh M. Patel, Raghu Ramakrishnan, and Cyrus Shahabi, “Big data and its technical
challenges,” IEEE Communications Magazine, vol. 57, no. 7, pp. 86–94, 2019.
K. Zoumpatianos and T. Palpanas. “Data Series Management: Fulfilling the Need for
Big Sequence Analytics”. In Proc. 34th Int. Conf. on Data Engineering, pp. 1677–1678.
IEEE, 2018.
M. Dreseler, M. Boissier, T. Rabl, and M. Uflacker, “Quantifying TPC-H choke points
and their optimizations,” Proc. VLDB Endowment, vol. 13, no. 8, pp. 1206–1220, 2020.
