Skip to content

NaDeSys/SparkDB

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

40 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SparkDB

GitHub forks GitHub stars GitHub issues GitHub issues
CSV-to-database-structure project that aims for:

  • write-on-memory connection instead of network connection
  • Simple usage and commands
  • Allowing data analysis for CSV raw data

Test it!

You can run $ javac main.java && java main to test SparkDB

Documentation

You can find javadoc here. A code example is src/main.java. See also doc/

Philosophy

There exists a lot of database solutions and types, such as (MY/MS)SQL, Redis, Postegres, etc.. . These database solutions are often an overkill for small projects and data manipulation. SparkDB is an abstract CSV-in-memory project, where every SparkDB object is a table. SparkDB offers a bloatware-free solution with an easy and minimal architecture.

With some coding, you can tame SparkDB to behave as a replacement for SQL, redis, or other database solutions. Write your own relational database, or in-memory cache with ease.

In short,

  • SparkDB is an abstract building block for database solutions.
  • SparkDB is used for scenarios where SQL-based/redis solutions are an overkill.
  • SparkDB does not have default Network-based communication, but it uses function calls.
  • SparkDB is NOT a relational database, but you can make your own relational implementation based on SparkDB.
  • SparkDB does not use a query language, but normal function calls that are understandable by any Java developer.
  • SparkDB is a minimal solution. Most of the time you have to use it as a building block to make a distributed database.
  • SparkDB only uses pure java. No frameworks or other libraries are required.

Note: SparkDB table has a maximum of 2.147 billion rows. If your usecase requires more rows, try to divide/shard the data.

Contributors

Morad Abdelrasheed - Youssef Hegazy (Indirect Suggestion, accepted) - Omar Mohamed Khallaf)