-
Notifications
You must be signed in to change notification settings - Fork 391
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] Scala 2.12 / Spark 3 upgrade #550
Open
nicodv
wants to merge
100
commits into
master
Choose a base branch
from
ndv/scala212
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+592
−1,757
Open
Changes from 1 commit
Commits
Show all changes
100 commits
Select commit
Hold shift + click to select a range
f6264a7
Update to Spark 2.4.3 and XGBoost 0.90
tovbinm 685d6e1
special double serializer fix
tovbinm e62772d
fix serialization
tovbinm 69247ac
fix serialization
tovbinm 330bf50
docs
tovbinm d6b0723
fixed missng value for test
wsuchy 63b77b5
meta fix
tovbinm 4e46e31
Merge branch 'mt/spark-2.4' of github.com:salesforce/TransmogrifAI in…
tovbinm 5a528e1
Updated DecisionTreeNumericMapBucketizer test to deal with the change…
Jauntbox 5f39603
Merge branch 'mt/spark-2.4' of github.com:salesforce/TransmogrifAI in…
Jauntbox 0d1a0c0
fix params meta test
tovbinm 0a4f906
FIxed failing xgboost test
wsuchy 660db62
Merge branch 'mt/spark-2.4' of github.com:salesforce/TransmogrifAI in…
wsuchy 3ecca64
ident
tovbinm 507503a
cleanup
tovbinm 348a392
added dataframe reader and writer extensions
tovbinm f43cb26
added const
tovbinm 4455034
Merge branch 'master' into mt/spark-2.4
tovbinm a0978bf
Merge branch 'master' into mt/spark-2.4
tovbinm 82aa188
build for scala 2.12
koertkuipers b27b47a
Merge branch 'master' of github.com:salesforce/TransmogrifAI into mt/…
tovbinm 6535e4e
added xgboost params + update models to use public predict method
tovbinm d1d7b9a
blarg
tovbinm ac75e15
double ser test
tovbinm 761b889
Merge remote-tracking branch 'upstream/mt/spark-2.4' into feat-scala212
koertkuipers 95095ed
fix unit tests by have lambdas implement concrete classes
koertkuipers 76b411b
Merge branch 'master' into feat-scala212
koertkuipers ecfb902
remove unnecessary method defaultMatches
koertkuipers a1a2579
Merge branch 'master' into feat-scala212
koertkuipers 785ddc5
Merge branch 'master' into feat-scala212
koertkuipers aacf00c
Merge branch 'master' into feat-scala212
koertkuipers c0a888f
Merge branch 'master' into feat-scala212
koertkuipers 9ececc9
use mleap release
koertkuipers 25a4449
Merge commit '51037a80ee6ef48c5c905ee967187288d78559cb' into feat-sca…
koertkuipers 53df597
Merge commit '95a77b17269a71bf0d53c54df7d76f0bfe862275' into feat-sca…
koertkuipers 4460fe5
Merge branch 'master' into feat-scala212
koertkuipers 5b29d8b
Merge branch 'master' into feat-scala212
koertkuipers 713a9f4
Merge branch 'master' into feat-scala212
koertkuipers 69a3678
Merge branch 'feat-scala212' of server02:oss/TransmogrifAI into feat-…
koertkuipers f4b3f01
Merge branch 'master' into feat-scala212
koertkuipers 142f121
Merge branch 'master' into feat-scala212
koertkuipers 5ee32b1
Merge branch 'master' into feat-scala212
koertkuipers 6e8e130
Merge branch 'feat-scala212' of server02:oss/TransmogrifAI into feat-…
koertkuipers c3ccdee
Merge branch 'master' into feat-scala212
koertkuipers ae1dfcf
Merge branch 'master' into feat-scala212
koertkuipers fd723d6
Increment scala hotfix prompted test change for random based doubles
tresata-gbernard e0f0bd8
Merge branch 'master' into feat-scala212
koertkuipers 98dafde
fix random numbers somehow being different in scala 2.12
koertkuipers accd2ba
Merge branch 'master' into feat-scala212
koertkuipers 27fdd3e
Merge branch 'master' into feat-scala212
koertkuipers f0cbc9e
WIP scala-multiversion-plugin
nicodv 7fb9f0a
Merge remote-tracking branch 'tresata/feat-scala212' into ndv/scala212
nicodv ff29d1b
upgrade xgboost to version that has 2.11 and 2.12 versions published
nicodv 20b8584
version string fixes
nicodv ca30345
add TODO
nicodv e2078e1
update TODO
nicodv fb16bd9
Merge branch 'master' into ndv/scala212
nicodv 5b61508
update version strings
nicodv 807eca9
update several versions to be scala 2.12 and spark 3 compatible
nicodv 3fba576
various compilation fixes
nicodv dc4adbc
stack is deprecated, use var List
nicodv 2cca254
use new udf interface
nicodv d3fbf8f
fix test
nicodv 9fbc9da
compilation fix
nicodv e8c5b7a
compilation fix
nicodv c61a5b7
deal with moved csv utils
nicodv 017676a
deal with deprecated operator
nicodv 0538892
disable test for now
nicodv 3e252db
add TODO
nicodv 4fe2fdf
Merge branch 'master' into ndv/scala212
tovbinm c1941e1
be explicit about xgboost4j dependency
nicodv bdfae00
Merge remote-tracking branch 'origin/ndv/scala212' into ndv/scala212
nicodv fe4f2fb
drop support for joined data readers and update docs accordingly
nicodv c649974
deal with deprecated operator
nicodv c391aac
refactor for Spark API changes to bin. class. metrics
nicodv 5f55dd9
use new 2.12 optimization options
nicodv 642d27c
adhere to new xgboost interface
nicodv 1605bd4
deal with deprecated syntax
nicodv 64ea9d2
update TODO
nicodv 51806fd
fix tree param overrides
crupley 09b2960
replace deprecated range with bigdecimal range
crupley a946ffb
Use public wrapper to SparkUserDefinedFunction (SparkUDFFactory) to g…
emitc2h ec7da39
update stack in while loop in FeatureLike.prettyParentStages
emitc2h 5b555e3
re-enabling @JSONdeserialize annotations while preserving the missing…
emitc2h 30e61a3
ensuring consistent behavior between FeatureDistribution equals and h…
emitc2h 9363b20
Merge branch 'master' into ndv/scala212
tovbinm 6f7c841
Added MomentsSerializer to allow json4s to serialize Algebird's Momen…
emitc2h 9a04faf
Merge branch 'ndv/scala212' of github.com:salesforce/TransmogrifAI in…
emitc2h 4f752ab
Fix random seed issues + coefficient ordering issues in ModelInsights
emitc2h 6731b9d
Fix expected results that changed due to changes in random number gen…
emitc2h b9e18ce
handle nulls and missing keys in cardinality calculations in SmartTex…
emitc2h c42163d
make test hash function consistent with OpHashingTF hashing (both now…
emitc2h 7082707
Don't shut down sparkContext after running a test suite, clear cache …
emitc2h 355bbe2
fixing unit tests in features
emitc2h 2cb1827
fixing unit test failures in testkit due to rng outcome changes
emitc2h fc5cdc8
Allow for some tolerance when comparing scores after model write/read…
emitc2h dc014fa
use legacy mode to read parquet files written with Spark 2.x (SPARK-3…
emitc2h f31ce9f
Store input schema column metadata in its own param during stage exec…
emitc2h 421b9bc
remove debug line
emitc2h 0038823
Rolling back most of the ColumnMetadata infra since inputSchema metad…
emitc2h File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tovbinm I just want to make sure this is correct. In principle hashCode equality and equals should be consistent and this is what I'm trying to accomplish here, but I figured you might have had a reason for wanting something different.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this test was invalid.