Skip to content

Commit

Permalink
Fixed bug with Sql Bulk Insert/Update processing with Model Propertie…
Browse files Browse the repository at this point in the history
…s that have mapped database names via mapping attribute (e.g. [SqlBulkColumn("")], [Map("")], [Column("")], etc.).
  • Loading branch information
cajuncoding committed Sep 2, 2023
1 parent fdde00f commit 0d9de38
Show file tree
Hide file tree
Showing 5 changed files with 84 additions and 8 deletions.
7 changes: 4 additions & 3 deletions NetStandard.SqlBulkHelpers/NetStandard.SqlBulkHelpers.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,19 @@
<PackageLicenseExpression>MIT</PackageLicenseExpression>
<Authors>BBernard / CajunCoding</Authors>
<Company>CajunCoding</Company>
<Version>2.3.0</Version>
<Version>2.3.1</Version>
<PackageProjectUrl>https://github.com/cajuncoding/SqlBulkHelpers</PackageProjectUrl>
<RepositoryUrl>https://github.com/cajuncoding/SqlBulkHelpers</RepositoryUrl>
<Description>A library for easy, efficient and high performance bulk insert and update of data, into a Sql Database, from .Net applications. By leveraging the power of the SqlBulkCopy classes with added support for Identity primary key table columns this library provides a greatly simplified interface to process Identity based Entities with Bulk Performance with the wide compatibility of .NetStandard 2.0.</Description>
<PackageTags>sql server database table bulk insert update identity column sqlbulkcopy orm dapper linq2sql materialization materialized data view materialized-data materialized-view sync replication replica readonly</PackageTags>
<PackageReleaseNotes>
- Fixed bug with Sql Bulk Insert/Update processing with Model Properties that have mapped database names via mapping attribute (e.g. [SqlBulkColumn("")], [Map("")], [Column("")], etc.).

Prior Relese Notes:
- Changed default behaviour to no longer clone tables/schema inside a Transaction which creates a full Schema Lock -- as this greatly impacts Schema aware ORMs such as SqlBulkHelpers, RepoDb, etc.
- New separate methods is now added to handle the CleanupMaterializeDataProcessAsync() but must be explicitly called as it is no longer implicitly called with FinishMaterializeDataProcessAsync().
- Added new configuration value to control if Schema copying/cloning (for Loading Tables) is inside or outide the Transaction (e.g. SchemaCopyMode.InsideTransactionAllowSchemaLocks vs OutsideTransactionAvoidSchemaLocks).
- Fix bug in ReSeedTableIdentityValueWithMaxIdAsync() when the Table is Empty so that it now defaults to value of 1.

Prior Relese Notes:
- Fixed a Bug where Identity Column Value was not correctly synced after Materialization Process is completing.
- Added new Helper API to quickly Sync the Identity column value with the current MAX Id value of the column (ensuring it's valid after populating additional data); This is useful if you override Identity values for a full Table refresh, but then want to later insert data into the table.
- Improved namespace for SqlBulkHelpers.CustomExtensions to reduce risk of conflicts with similar existing extensions.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ public bool Read()
return _dataEnumerator.MoveNext();
}

public int GetOrdinal(string name)
public int GetOrdinal(string dbColumnName)
{
//Lazy Load the Ordinal reverse lookup dictionary (ONLY if needed)
if (_processingDefinitionOrdinalDictionary == null)
Expand All @@ -58,15 +58,15 @@ public int GetOrdinal(string name)
int i = 0;
_processingDefinitionOrdinalDictionary = new Dictionary<string, int>();
foreach (var propDef in _processingFields)
_processingDefinitionOrdinalDictionary[propDef.PropertyName] = i++;
_processingDefinitionOrdinalDictionary[propDef.MappedDbColumnName] = i++;
}

if (SqlBulkHelpersConstants.ROWNUMBER_COLUMN_NAME.Equals(name))
if (SqlBulkHelpersConstants.ROWNUMBER_COLUMN_NAME.Equals(dbColumnName))
return _rowNumberPseudoColumnOrdinal;
else if (_processingDefinitionOrdinalDictionary.TryGetValue(name, out var ordinalIndex))
else if (_processingDefinitionOrdinalDictionary.TryGetValue(dbColumnName, out var ordinalIndex))
return ordinalIndex;

throw new ArgumentOutOfRangeException($"Property name [{name}] could not be found.");
throw new ArgumentOutOfRangeException($"Property name [{dbColumnName}] could not be found.");
}

public object GetValue(int i)
Expand Down
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,16 @@ public class TestDataService
## Nuget Package
To use in your project, add the [SqlBulkHelpers NuGet package](https://www.nuget.org/packages/SqlBulkHelpers/) to your project.

## v2.3.1 Release Notes:
- Fixed bug with Sql Bulk Insert/Update processing with Model Properties that have mapped database names via mapping attribute (e.g. [SqlBulkColumn("")], [Map("")], [Column("")], etc.).

## v2.3 Release Notes:
- Changed default behaviour to no longer clone tables/schema inside a Transaction which creates a full Schema Lock -- as this greatly impacts Schema aware ORMs such as SqlBulkHelpers, RepoDb, etc.
- Note: If you are manually orchestrating your process using StartMaterializedDataProcessAsync() and FinishMaterializeDataProcessAsync() then you now need to handle this by explicitly calling CleanupMaterializeDataProcessAsync() in a Try/Finally.
- Added new configuration value to control if Schema copying/cloning (for Loading Tables) is inside or outide the Transaction (e.g. SchemaCopyMode.InsideTransactionAllowSchemaLocks vs OutsideTransactionAvoidSchemaLocks).
- Fix bug in ReSeedTableIdentityValueWithMaxIdAsync() when the Table is Empty so that it now defaults to value of 1.


## v2.2.2 Release Notes:
- Improved namespace for SqlBulkHelpers.CustomExtensions to reduce risk of conflicts with similar existing extensions.

Expand Down
12 changes: 12 additions & 0 deletions SqlBulkHelpers.SampleApp.Common/TestHelpers.cs
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,18 @@ public class ChildTestElement
[SqlBulkTable(TestHelpers.TestTableName, uniqueMatchMergeValidationEnabled: false)]
public class TestElementWithMappedNames
{
public TestElementWithMappedNames()
{
}

public TestElementWithMappedNames(TestElement testElement)
{
MyId = testElement.Id;
MyKey = testElement.Key;
MyValue = testElement.Value;
UnMappedProperty = -1;
}

[SqlBulkMatchQualifier]
[Map("Id")]
public int MyId { get; set; }
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -141,5 +141,58 @@ public async Task TestBulkInsertOrUpdateWithMultipleCustomMatchQualifiersAsync()
}
}
}

[TestMethod]
public async Task TestBulkInsertOrUpdateWithMappedPropertiesAndCustomMatchQualifiersAsync()
{
var testDataWithMappedProps = TestHelpers
.CreateTestData(10)
.Select(t =>
{
var r = new TestElementWithMappedNames(t);
r.MyKey = $"MULTIPLE_QUALIFIER_TEST-{r.MyKey}";
return r;
})
.ToList();

var sqlConnectionString = SqlConnectionHelper.GetSqlConnectionString();
ISqlBulkHelpersConnectionProvider sqlConnectionProvider = new SqlBulkHelpersConnectionProvider(sqlConnectionString);

await using var sqlConn = await sqlConnectionProvider.NewConnectionAsync().ConfigureAwait(false);
await using (var sqlTrans = (SqlTransaction)await sqlConn.BeginTransactionAsync().ConfigureAwait(false))
{
var results = await sqlTrans.BulkInsertOrUpdateAsync(
testDataWithMappedProps,
TestHelpers.TestTableName,
new SqlMergeMatchQualifierExpression(
nameof(TestElement.Value),
nameof(TestElement.Key) //This will still result in UNIQUE entries that are being inserted (NO UPDATES)
)
);

await sqlTrans.CommitAsync().ConfigureAwait(false);

//ASSERT Results are Valid...
Assert.IsNotNull(results);

//We Sort the Results by Identity Id to ensure that the inserts occurred in the correct
// ordinal order matching our Array of original values, but now with incrementing ID values!
//This validates that data is inserted as expected for Identity columns and is validated
// correctly by sorting on the Incrementing Identity value when Queried (e.g. ORDER BY Id)
// which must then match our original order of data.
var resultsSorted = results.OrderBy(r => r.MyId).ToList();
Assert.AreEqual(resultsSorted.Count(), testDataWithMappedProps.Count);

var i = 0;
foreach (var result in resultsSorted)
{
Assert.IsNotNull(result);
Assert.IsTrue(result.MyId > 0);
Assert.AreEqual((object)result.MyKey, testDataWithMappedProps[i].MyKey);
Assert.AreEqual((object)result.MyValue, testDataWithMappedProps[i].MyValue);
i++;
}
}
}
}
}

0 comments on commit 0d9de38

Please sign in to comment.