Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/psql mariadb support #214

Merged
merged 51 commits into from
Sep 28, 2023
Merged
Show file tree
Hide file tree
Changes from 43 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
57c8c66
Add PSQL-compatible schema/DDL.
Sehkah Sep 17, 2023
254d67b
Add simple PSQL config.
Sehkah Sep 17, 2023
6a957bf
Add compatible PSQL dependency.
Sehkah Sep 17, 2023
5786ffd
Add PSQL database implementation & choice.
Sehkah Sep 17, 2023
e2bed79
Add PSQL-specific configurations and docker compose setup.
Sehkah Sep 17, 2023
4c3bf23
Replace backticks in queries with escaped double quotation mark for P…
Sehkah Sep 17, 2023
78a1dee
Fix UTC-related timezone issues with PSQL.
Sehkah Sep 17, 2023
21e24d8
Add initial required setup for MariaDB support.
Sehkah Sep 17, 2023
358b4db
Fix PSQL schema-related issues by migrating SMALLINT to INTEGER, BIT …
Sehkah Sep 18, 2023
389d7ba
Fix PSQL ddon_item & ddon_character_job_data insert.
Sehkah Sep 18, 2023
2adbb62
Add named primary key constraints to PSQL schema.
Sehkah Sep 19, 2023
363793b
Expose incompatible SQLite queries as virtual properties to allow for…
Sehkah Sep 19, 2023
179d8a3
Implement PSQL-specific inserts.
Sehkah Sep 19, 2023
597a294
Fix PSQL-related time zone issue with DateTime.
Sehkah Sep 19, 2023
9028d72
Migrate PSQL schema back to smallint to deal with numeric type persis…
Sehkah Sep 19, 2023
49b2a2d
Support login process with MariaDB.
Sehkah Sep 19, 2023
8ab33be
Initial working state for MariaDB.
Sehkah Sep 19, 2023
f7710f8
Fix PSQL time zone & ushort/uint issues.
Sehkah Sep 21, 2023
a2c2d44
Reuse DB connections unless a new connection is actually needed.
Sehkah Sep 21, 2023
970844f
Extract common logic for reusable connections.
Sehkah Sep 21, 2023
7cb6e3a
Open new logical DB connections by default.
Sehkah Sep 21, 2023
d6de097
Reset default config json.
Sehkah Sep 21, 2023
8ccb17f
Reduce differences between database schemas.
Sehkah Sep 22, 2023
0d6a23a
Further reduce differences between database schemas.
Sehkah Sep 22, 2023
153320d
Support defining database type as friendly string.
Sehkah Sep 22, 2023
7533f89
Use alternative, generic sub-query approach to implement SQL "ignore"…
Sehkah Sep 23, 2023
1a1926d
Use SQLite schema as foundation & transform incompatible syntax to av…
Sehkah Sep 23, 2023
bf6b552
Fix schema file reference.
Sehkah Sep 23, 2023
58fafce
Ensure identification of db enum type happens after JSON serializatio…
Sehkah Sep 23, 2023
a00e2b8
Migrate DB logic to replace character job data to be more engine agno…
Sehkah Sep 24, 2023
4eb7eb6
Migrate DB logic to replace stored items to be more engine agnostic.
Sehkah Sep 24, 2023
e225d08
Migrate DB logic to replace storage to be more engine agnostic.
Sehkah Sep 24, 2023
19d8397
Migrate DB logic to replace communication shortcuts to be more engine…
Sehkah Sep 24, 2023
ffbdff4
Migrate DB logic to replace equipped items to be more engine agnostic.
Sehkah Sep 24, 2023
b8cc798
Migrate DB logic to replace equipped job items to be more engine agno…
Sehkah Sep 24, 2023
3d6ba94
Migrate DB logic to replace equipped custom skills to be more engine …
Sehkah Sep 25, 2023
34e5063
Migrate DB logic to replace wallet points to be more engine agnostic.
Sehkah Sep 25, 2023
84ca96f
Migrate DB logic to replace shortcuts to be more engine agnostic.
Sehkah Sep 25, 2023
0efb0fb
Migrate DB logic to replace normal skill parameters to be more engine…
Sehkah Sep 25, 2023
fece791
Migrate DB logic to replace sp skills to be more engine agnostic.
Sehkah Sep 25, 2023
2a111b5
Migrate DB logic to replace pawn reactions to be more engine agnostic.
Sehkah Sep 25, 2023
bec2c0c
Migrate DB logic to replace equipped ability to be more engine agnostic.
Sehkah Sep 25, 2023
df7c322
Avoid duplicate definitions of db parameters.
Sehkah Sep 25, 2023
5be89d4
Make schema string replacements more robust.
Sehkah Sep 25, 2023
55fa596
Fix mariadb healthcheck.
Sehkah Sep 26, 2023
979b4c5
Ensure environment variables are loaded in case the database settings…
Sehkah Sep 26, 2023
e52ff73
Extract DB parameters to environment variables for sample docker setup.
Sehkah Sep 26, 2023
259cb98
Update readme.
Sehkah Sep 26, 2023
3a87b50
Add build badge to readme.
Sehkah Sep 26, 2023
030badf
Extract runtime identifier as Docker build argument to better support…
Sehkah Sep 27, 2023
92eaaa4
Default WipeOnStartup to false
alborrajo Sep 28, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Arrowgene.Ddon.Database/Arrowgene.Ddon.Database.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@
<ItemGroup>
<PackageReference Include="Arrowgene.Logging" Version="1.2.1" />
<PackageReference Include="System.Data.SQLite" Version="1.0.115.5" />
<PackageReference Include="Npgsql" Version="7.0.6" />
<PackageReference Include="MySqlConnector" Version="2.2.7" />
</ItemGroup>

<ItemGroup>
Expand Down
46 changes: 33 additions & 13 deletions Arrowgene.Ddon.Database/DatabaseSetting.cs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
using System.Runtime.Serialization;
using Arrowgene.Ddon.Database.Model;
using Arrowgene.Ddon.Shared;
using Arrowgene.Ddon.Shared.Network;

namespace Arrowgene.Ddon.Database
{
Expand All @@ -12,8 +11,8 @@ public class DatabaseSetting
{
public DatabaseSetting()
{
Type = DatabaseType.SQLite;
SqLiteFolder = Path.Combine(Util.ExecutingDirectory(), "Files/Database");
Type = "sqlite";
DatabaseFolder = Path.Combine(Util.ExecutingDirectory(), "Files/Database");
Host = "localhost";
Port = 3306;
Database = "Ddon";
Expand All @@ -22,30 +21,51 @@ public DatabaseSetting()
WipeOnStartup = true;

string envDbType = Environment.GetEnvironmentVariable("DB_TYPE");
switch (envDbType)
if (!string.IsNullOrEmpty(envDbType))
{
case "sqlite":
Type = DatabaseType.SQLite;
break;
Type = envDbType;
}
string envDbFolder = Environment.GetEnvironmentVariable("DB_FOLDER");
if (!string.IsNullOrEmpty(envDbFolder))
{
DatabaseFolder = envDbFolder;
}
string envDbHost = Environment.GetEnvironmentVariable("DB_HOST");
if (!string.IsNullOrEmpty(envDbHost))
{
Host = envDbHost;
}
string envDbPort = Environment.GetEnvironmentVariable("DB_PORT");
if (!string.IsNullOrEmpty(envDbPort))
{
Port = Convert.ToInt16(envDbPort);
}
string envDbDatabase = Environment.GetEnvironmentVariable("DB_DATABASE");
if (!string.IsNullOrEmpty(envDbDatabase))
{
Database = envDbDatabase;
}

string envDbUser = Environment.GetEnvironmentVariable("DB_USER");
if (!string.IsNullOrEmpty(envDbUser))
{
User = envDbUser;
}

string envDbPass = Environment.GetEnvironmentVariable("DB_PASS");
if (!string.IsNullOrEmpty(envDbPass))
{
Password = envDbPass;
}
string envDbWipeOnStartup = Environment.GetEnvironmentVariable("DB_WIPE_ON_STARTUP");
if (!string.IsNullOrEmpty(envDbWipeOnStartup))
{
WipeOnStartup = Convert.ToBoolean(envDbWipeOnStartup);
}
}

public DatabaseSetting(DatabaseSetting databaseSettings)
{
Type = databaseSettings.Type;
SqLiteFolder = databaseSettings.SqLiteFolder;
DatabaseFolder = databaseSettings.DatabaseFolder;
Host = databaseSettings.Host;
Port = databaseSettings.Port;
User = databaseSettings.User;
Expand All @@ -54,9 +74,9 @@ public DatabaseSetting(DatabaseSetting databaseSettings)
WipeOnStartup = databaseSettings.WipeOnStartup;
}

[DataMember(Order = 0)] public DatabaseType Type { get; set; }
[DataMember(Order = 0)] public string Type { get; set; }

[DataMember(Order = 1)] public string SqLiteFolder { get; set; }
[DataMember(Order = 1)] public string DatabaseFolder { get; set; }

[DataMember(Order = 2)] public string Host { get; set; }

Expand All @@ -68,6 +88,6 @@ public DatabaseSetting(DatabaseSetting databaseSettings)

[DataMember(Order = 6)] public string Database { get; set; }

[DataMember(Order = 6)] public bool WipeOnStartup { get; set; }
[DataMember(Order = 7)] public bool WipeOnStartup { get; set; }
}
}
71 changes: 51 additions & 20 deletions Arrowgene.Ddon.Database/DdonDatabaseBuilder.cs
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
using System;
using System.IO;
using System.Text;
using System.Text.RegularExpressions;
using Arrowgene.Ddon.Database.Model;
using Arrowgene.Ddon.Database.Sql;
using Arrowgene.Logging;
Expand All @@ -9,46 +11,75 @@ namespace Arrowgene.Ddon.Database
public static class DdonDatabaseBuilder
{
private static readonly ILogger Logger = LogProvider.Logger<Logger>(typeof(DdonDatabaseBuilder));
private const string DefaultSchemaFile = "Script/schema_sqlite.sql";

public static IDatabase Build(DatabaseSetting settings)
{
IDatabase database = null;
switch (settings.Type)
Enum.TryParse(settings.Type, true, out DatabaseType dbType);
IDatabase database = dbType switch
{
case DatabaseType.SQLite:
string sqLitePath = Path.Combine(settings.SqLiteFolder, $"db.v{DdonSqLiteDb.Version}.sqlite");
database = BuildSqLite(settings.SqLiteFolder, sqLitePath, settings.WipeOnStartup);
break;
}
DatabaseType.SQLite => BuildSqLite(settings.DatabaseFolder, settings.WipeOnStartup),
DatabaseType.PostgreSQL => BuildPostgres(settings.DatabaseFolder, settings.Host, settings.User, settings.Password, settings.Database, settings.WipeOnStartup),
DatabaseType.MariaDb => BuildMariaDB(settings.DatabaseFolder, settings.Host, settings.User, settings.Password, settings.Database, settings.WipeOnStartup),
_ => throw new ArgumentOutOfRangeException($"Unknown database type '{settings.Type}' encountered!")
};

if (database == null)
{
Logger.Error("Database could not be created, exiting...");
Environment.Exit(1);
}
else
{
Logger.Info($"Database of type '${dbType.ToString()}' has been created.");
Logger.Info($"Database path: {settings.DatabaseFolder}");
}

return database;
}

public static DdonSqLiteDb BuildSqLite(string sqLiteFolder, string sqLitePath, bool deleteIfExists)
public static DdonSqLiteDb BuildSqLite(string databaseFolder, bool wipeOnStartup)
{
string sqLitePath = Path.Combine(databaseFolder, $"db.v{DdonSqLiteDb.Version}.sqlite");
DdonSqLiteDb db = new DdonSqLiteDb(sqLitePath, wipeOnStartup);
if (db.CreateDatabase())
{
string schemaFilePath = Path.Combine(databaseFolder, DefaultSchemaFile);
String schema = File.ReadAllText(schemaFilePath, Encoding.UTF8);

db.Execute(schema);
}

return db;
}

public static DdonPostgresDb BuildPostgres(string databaseFolder, string host, string user, string password, string database, bool wipeOnStartup)
{
if (deleteIfExists)
DdonPostgresDb db = new DdonPostgresDb(host, user, password, database, wipeOnStartup);
if (db.CreateDatabase())
{
try
{
File.Delete(sqLitePath);
}
catch (Exception)
{
// ignored
}
string schemaFilePath = Path.Combine(databaseFolder, DefaultSchemaFile);
String schema = File.ReadAllText(schemaFilePath, Encoding.UTF8);
schema = Regex.Replace(schema, "(\\s)DATETIME(\\s)", " TIMESTAMP WITH TIME ZONE ");
schema = Regex.Replace(schema, "(\\s)INTEGER PRIMARY KEY AUTOINCREMENT(\\s)", " SERIAL PRIMARY KEY ");
schema = Regex.Replace(schema, "(\\s)BLOB(\\s)", " BYTEA ");

db.Execute(schema);
}

DdonSqLiteDb db = new DdonSqLiteDb(sqLitePath);
return db;
}

public static DdonMariaDb BuildMariaDB(string databaseFolder, string host, string user, string password, string database, bool wipeOnStartup)
{
DdonMariaDb db = new DdonMariaDb(host, user, password, database, wipeOnStartup);
if (db.CreateDatabase())
{
ScriptRunner scriptRunner = new ScriptRunner(db);
scriptRunner.Run(Path.Combine(sqLiteFolder, "Script/schema_sqlite.sql"));
string schemaFilePath = Path.Combine(databaseFolder, DefaultSchemaFile);
String schema = File.ReadAllText(schemaFilePath, Encoding.UTF8);
schema = Regex.Replace(schema, "(\\s)AUTOINCREMENT(\\s)", " AUTO_INCREMENT ");

db.Execute(schema);
}

return db;
Expand Down
Loading
Loading