dkeeshin
Funding Links: https://github.com/sponsors/dkeeshin
- Location: Evanston IL
- Company: Keeshin Database Services , LLC (kDS)
- Kind: user
- Followers: 0
- Following: 0
- Total stars: 2
- Repositories count: 5
- Created at: 2022-11-18T13:30:15.043Z
- Updated at: 2025-04-24T05:49:21.692Z
- Last synced at: 2025-04-24T05:49:21.692Z
GitHub Sponsors Profile
For the past year I have been conducting data technology research and development work. I have blogged about this work at http://keeshinds.com/blog/kDSBlog.html.
I covered topics such as:
Developing a data management solution for tracking COVID 19 data.
Using a data vault design strategy.
Reviewing popular APIs.
Evaluating open-source software.
This work has been funded by my company Keeshin Database Services, LLC(kDS) and the Small Business Administration (SBA).
Recently, a colleague of mine and I discussed the challenge of building a peer-to-peer network of databases. Thinking about my recent research and writing, I started looking at ways to do that. After a few "ah-ha" moments, I came up with this concept.
OLTP 2.0
I call it “OLTP 2.0”, an open source, peer to peer OLTP "network of databases" concept using best of breed components, Linux, PostgreSQL, GO(GOLANG) and gRPC.
PostgreSQL is the engine that drives the entire process. It asynchronously communicates to all databases that are participating in the peer-to-peer network. I selected PostgreSQL because it is powerful, reliable, open-source object-relational database system. The origins of PostgreSQL date back to 1986. It has been ACID compliant since 2001. It has built-in asynchronous capabilities like LISTEN and NOTIFY as well as functions and stored procedures that simplify managing data in a peer-to-peer setting.
I use GO (or GOLANG as it is often called) to “listen” for data messages sent from PostgreSQL. Messages are then sent over a gRPC connection to a remote peer database. GO was selected because of its static typing, run-time efficiency, readability, usability, and high-performance networking and multiprocessing features. It also integrates with gRPC.
gRPC uses protocol buffers for sending and receiving non-readable, binary data messages. This eliminates the need to serialize/de-serialize XML and JSON data like REST or SOAP APIs require. It also up to ten times faster.
gRPC requires HTTP/2. Introduced in 2015, HTTP/2 is a major revision of the HTTP1.1 transport protocol. All current versions of popular web browsers now support it.
In addition to GO, gRPC supports multiple languages including: C#/.NET, C++, Dart, Java, Kotlin/JVM, Node.js, Objective-C, PHP, Python, and Ruby.
Below is the GitHub repository containing proof-of-concept code and documentation I have created to date.
SPONSORSHIP
Going forward, sponsorship for this work is critical. It would allow me to:
Develop a detailed system design.
Build out a full working model and test scaling out to 100 or even 1000 peers.
Evaluate data distribution approaches for handling large data sets.
Work with the open-source community and experts to collaborate, develop and promote this approach.
Identify proof of concept candidates.
Help pay loans that funded this work to date.
- Current Sponsors: 0
- Past Sponsors: 0
- Total Sponsors: 0
- Minimum Sponsorship: $350.00
Featured Works
dkeeshin/OLTP20_framework
An open source, peer-to-peer OLTP "network of databases" concept using best of breed components, Linux, PostgreSQL, GO(GOLANG) and gRPC.
Language: PLpgSQL - Stars: