-
Notifications
You must be signed in to change notification settings - Fork 662
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CircleCI to GHA migration #7154
Merged
Merged
Changes from 154 commits
Commits
Show all changes
155 commits
Select commit
Hold shift + click to select a range
266c412
Disable CircleCI
gokhangulbiz 50dd50f
Flaky test debugging on GHA
gokhangulbiz f5c680f
Reorder required params.
gokhangulbiz b3adf9c
Reorganize env variables and inputs.
gokhangulbiz 48a987e
Reorganize upload artifact folder structure
gokhangulbiz 2c9375b
Initial commit for citus testing on GHA
gokhangulbiz 488271f
Fix image suffix.
gokhangulbiz 13b62fe
Fix file path
gokhangulbiz 1d21f50
Fix parameter binding
gokhangulbiz 6f883f3
Use root user for container images
gokhangulbiz a5a65cc
Fix container image suffix
gokhangulbiz 80db3f1
naming
gokhangulbiz 58da0c5
Artifact upload optimization
gokhangulbiz 204df4b
download artifact path fix
gokhangulbiz acc18ec
download artifact path fix
gokhangulbiz 6005499
Fix invalid attribute
gokhangulbiz fbe7675
Fix downloaded artifacts folder
gokhangulbiz 0ed6ea1
Minor fixes
gokhangulbiz a52e4a0
Fix artifacts path
gokhangulbiz 9c47dba
Matrix customization for failtester
gokhangulbiz 91e3229
Stack trace and upload covarage run always
gokhangulbiz 01e2596
Fix for check-sql-snapshots and check-style issues
gokhangulbiz c1ff845
Run container image with root user
gokhangulbiz 812077e
Change default user to circleci
gokhangulbiz cf62647
Safe directory configuration
gokhangulbiz 6127fe3
Fix newline issues
gokhangulbiz 0ba0c68
Set user to root for sql snapshots
gokhangulbiz f4bc979
Fetch all branches to be able to run git diff..
gokhangulbiz e5b5581
Merge remote-tracking branch 'origin/main' into circleci-gha-migration
gokhangulbiz 97a8a31
Include check-failure in matrix
gokhangulbiz 0fd2e9b
Reorganize failtester matrix
gokhangulbiz 0e3a292
Use env variable for PG_MAJOR
gokhangulbiz 20f8baa
Do not override PG_MAJOR
gokhangulbiz c77306d
Try to evaluate env variable case-sensitivity
gokhangulbiz 4dd0a8a
Test
gokhangulbiz 5afb9a1
Add PG_MAJOR as Github Env Var
gokhangulbiz f14938a
Add PG_MAJOR as Github Env Var
gokhangulbiz fb937db
Add PG_MAJOR as Github Env Var
gokhangulbiz 1bc08df
Naming
gokhangulbiz f028827
Env var fix
gokhangulbiz c0a1998
Update matrix
gokhangulbiz c9c2d87
Update matrix
gokhangulbiz 59dbf06
cdc and tap tests
gokhangulbiz 5626f58
Fix log processing
gokhangulbiz f655f6d
Set timeout
gokhangulbiz 451d6d8
fix step summary
gokhangulbiz 1363824
fix wildcard path
gokhangulbiz 903196c
minor fixes
gokhangulbiz 13ceea3
narrow down upload path
gokhangulbiz f67f468
better wildcard usage
gokhangulbiz 609fbc8
Diff markdown
gokhangulbiz 088d70a
Cleanup
gokhangulbiz 7d7f282
Fix double quote
gokhangulbiz 3a40fce
Naming refactorings
gokhangulbiz 16d03c9
Cleanup
gokhangulbiz c1904c4
Cleanup
gokhangulbiz dd088d3
Refactoring
gokhangulbiz 48ec1b8
test-arbitrary-configs
gokhangulbiz b7e6eff
check-query-generator and pgupgrade
gokhangulbiz 5347a51
fix
gokhangulbiz 9419b4b
fix parallelization
gokhangulbiz eb73c20
Parallelism
gokhangulbiz 072d7be
Remove workflow env variables.
gokhangulbiz a9c4bb5
Remove empty new line
gokhangulbiz 16a55cb
test-citus-upgrade
gokhangulbiz 5c4852d
upload-coverage
gokhangulbiz 698a5f0
ch_benchmark
gokhangulbiz 114574b
tpcc_benchmark
gokhangulbiz 76196d5
test-flakyness
gokhangulbiz d2d7907
Fix test-arbitrary-config
gokhangulbiz 0a63f31
fix test-flakyness
gokhangulbiz c46eeb2
Fix test-citus-upgrade
gokhangulbiz 2f65a2c
Fix test-arbitrary-configs
gokhangulbiz de7d896
Enable test-citus
gokhangulbiz a80186d
Fix install_extension
gokhangulbiz eb6cb48
Merge remote-tracking branch 'origin/main' into circleci-gha-migration
gokhangulbiz 291924f
Merge branch 'main' of https://github.com/citusdata/citus
gokhangulbiz d877f2d
Fix install extension
gokhangulbiz b691fc0
Fix install extension
gokhangulbiz ac17f07
Fix install extension
gokhangulbiz 5f0f5cd
Fix install extension
gokhangulbiz ef20671
Fix install extension
gokhangulbiz 035c6da
minor fixes
gokhangulbiz 0af5f40
Minor updates
gokhangulbiz 4aab137
fixe upgrade tests
gokhangulbiz 27b47a2
Path fix
gokhangulbiz f4a358e
Fix path and permissions
gokhangulbiz e3689c5
Fix test-arbitrary-configs
gokhangulbiz 95c9d26
skip_installation for test-citus-upgrade
gokhangulbiz abe4c42
fix condition
gokhangulbiz 2ed1e76
Cleanup
gokhangulbiz 5fe3ae7
16core vm
gokhangulbiz 7779d89
Fix condition check
gokhangulbiz 601a5f0
32core vm for build
gokhangulbiz 2a6dbc6
32core vm for test-citus
gokhangulbiz e39a421
16core
gokhangulbiz 1774385
revert 16core vm
gokhangulbiz e0fab47
Disable test-arbitrary-configs till we have 16core VMs
gokhangulbiz 262aff2
Improve diff output
gokhangulbiz ae588a3
Adds concurrency
gokhangulbiz 7b1c662
Fix for diffs output
gokhangulbiz b22ecb0
typo fix
gokhangulbiz 784169c
flakyness test
gokhangulbiz d33088a
test-flakyness container image fix
gokhangulbiz 2398b2f
Skip_test_flakyness added.
gokhangulbiz a6ec872
Move workflow inputs to repository variables
gokhangulbiz bb9bdf7
Cleanup
gokhangulbiz 509c173
Fix test-flakyness
gokhangulbiz ad18d5d
Always save logs and results
gokhangulbiz e7d75f0
Fix container image name
gokhangulbiz 3a76759
Add concurrency for flaky
gokhangulbiz dd8dfb9
Improve run name for build and test
gokhangulbiz cfa58e9
fix if condition
gokhangulbiz fa38db2
Preserve newlines
gokhangulbiz ca8aa3f
Fix condition
gokhangulbiz 16f360a
better naming
gokhangulbiz f92c0fa
Revert "flakyness test"
gokhangulbiz fdc35c4
Provide path for codeclimate results
gokhangulbiz 818bd9c
Disable test-citus temporarily
gokhangulbiz 04c9d07
Fix upload coverage
gokhangulbiz a73da25
fix upload coverage
gokhangulbiz d5f1747
fix upload coverage
gokhangulbiz 88c92cc
codecov and codeclimate
gokhangulbiz 1c39c8b
Revert "Disable test-citus temporarily"
gokhangulbiz b291b09
Enable PR triggers
gokhangulbiz a3a6df2
Additional configuration for codecov
gokhangulbiz f961ed8
Remove run-name for build task
gokhangulbiz 5c5375e
Fix secrets access in a composite action
gokhangulbiz 752eaff
Better naming work build and test
gokhangulbiz 142a631
enable gcov
gokhangulbiz 82c97a0
Renaming
gokhangulbiz b46ac3f
Merge pull request #1 from gokhangulbiz/circleci-gha-migration-pr-tri…
gokhangulbiz c272bb5
Merge branch 'main', remote-tracking branch 'origin' into circleci-gh…
gokhangulbiz 2493890
Merge branch 'main' into circleci-gha-migration
gokhangulbiz 7a5325f
Merge branch 'main' into circleci-gha-migration
gokhangulbiz 2700fb9
Merge branch 'main' into circleci-gha-migration
gokhangulbiz daa327d
Add PG16 to matrix.
gokhangulbiz 9d41b76
Use pg14 for citus tests.
gokhangulbiz ca96868
Merge branch 'main' into circleci-gha-migration
gokhangulbiz e94a700
Remove duplicated action."
gokhangulbiz b498956
modify echo flag.
gokhangulbiz adae8b6
Merge branch 'circleci-gha-migration' of https://github.com/citusdata…
gokhangulbiz ce9da5e
Use ubuntu-22.04
gokhangulbiz 5c3c9a7
1ES Hosted pool runners for build and arbitrary config tests
gokhangulbiz fbe81bc
Preserve new lines for regression outputs
gokhangulbiz 6577272
debug - open 57637 port
gokhangulbiz 3ec61f9
Debug - use public nameserver to have similar network setup with Circ…
gokhangulbiz 4fad9b6
Merge branch 'main' into circleci-gha-migration
gokhangulbiz 25486bb
Add more comments on custom DNS settings.
gokhangulbiz f5b7d5b
Use github hosted runners for build jobs.
gokhangulbiz 2563376
Provide a workaround for test splitting in arbitrary configs tests.
gokhangulbiz 8b14839
Improve namings and outputs
gokhangulbiz 80b7909
Enable CircleCI
gokhangulbiz 0d3fa0b
Merge branch 'main' into circleci-gha-migration
gokhangulbiz ae3478f
Merge branch 'main' into circleci-gha-migration
hanefi File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
name: 'Parallelization matrix' | ||
inputs: | ||
count: | ||
required: false | ||
default: 32 | ||
outputs: | ||
json: | ||
value: ${{ steps.generate_matrix.outputs.json }} | ||
runs: | ||
using: "composite" | ||
steps: | ||
- name: Generate parallelization matrix | ||
id: generate_matrix | ||
shell: bash | ||
run: |- | ||
json_array="{\"include\": [" | ||
for ((i = 1; i <= ${{ inputs.count }}; i++)); do | ||
json_array+="{\"id\":\"$i\"}," | ||
done | ||
json_array=${json_array%,} | ||
json_array+=" ]}" | ||
echo "json=$json_array" >> "$GITHUB_OUTPUT" | ||
echo "json=$json_array" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,38 @@ | ||||||
name: save_logs_and_results | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why do we not have a human-readable name instead?
Suggested change
I assumed quotation marks are not necessary, but I am not so sure about that. |
||||||
inputs: | ||||||
folder: | ||||||
required: false | ||||||
default: "log" | ||||||
runs: | ||||||
using: composite | ||||||
steps: | ||||||
- uses: actions/upload-artifact@v3.1.1 | ||||||
name: Upload logs | ||||||
with: | ||||||
name: ${{ inputs.folder }} | ||||||
if-no-files-found: ignore | ||||||
path: | | ||||||
src/test/**/proxy.output | ||||||
src/test/**/results/ | ||||||
src/test/**/tmp_check/master/log | ||||||
src/test/**/tmp_check/worker.57638/log | ||||||
src/test/**/tmp_check/worker.57637/log | ||||||
src/test/**/*.diffs | ||||||
src/test/**/out/ddls.sql | ||||||
src/test/**/out/queries.sql | ||||||
src/test/**/logfile_* | ||||||
/tmp/pg_upgrade_newData_logs | ||||||
- name: Publish regression.diffs | ||||||
run: |- | ||||||
diffs="$(find src/test/regress -name "*.diffs" -exec cat {} \;)" | ||||||
if ! [ -z "$diffs" ]; then | ||||||
echo '```diff' >> $GITHUB_STEP_SUMMARY | ||||||
echo -E "$diffs" >> $GITHUB_STEP_SUMMARY | ||||||
echo '```' >> $GITHUB_STEP_SUMMARY | ||||||
echo -E $diffs | ||||||
fi | ||||||
shell: bash | ||||||
- name: Print stack traces | ||||||
run: "./ci/print_stack_trace.sh" | ||||||
if: failure() | ||||||
shell: bash |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
name: setup_extension | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why do we not have a human-readable name here? |
||
inputs: | ||
pg_major: | ||
required: false | ||
skip_installation: | ||
required: false | ||
default: false | ||
type: boolean | ||
runs: | ||
using: composite | ||
steps: | ||
- name: Expose $PG_MAJOR to Github Env | ||
run: |- | ||
if [ -z "${{ inputs.pg_major }}" ]; then | ||
echo "PG_MAJOR=${PG_MAJOR}" >> $GITHUB_ENV | ||
else | ||
echo "PG_MAJOR=${{ inputs.pg_major }}" >> $GITHUB_ENV | ||
fi | ||
shell: bash | ||
- uses: actions/download-artifact@v3.0.1 | ||
with: | ||
name: build-${{ env.PG_MAJOR }} | ||
- name: Install Extension | ||
if: ${{ inputs.skip_installation == 'false' }} | ||
run: tar xfv "install-$PG_MAJOR.tar" --directory / | ||
shell: bash | ||
- name: Configure | ||
run: |- | ||
chown -R circleci . | ||
git config --global --add safe.directory ${GITHUB_WORKSPACE} | ||
gosu circleci ./configure --without-pg-version-check | ||
shell: bash | ||
- name: Enable core dumps | ||
run: ulimit -c unlimited | ||
shell: bash |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
name: coverage | ||
inputs: | ||
flags: | ||
required: false | ||
codecov_token: | ||
required: true | ||
runs: | ||
using: composite | ||
steps: | ||
- uses: codecov/codecov-action@v3 | ||
with: | ||
flags: ${{ inputs.flags }} | ||
token: ${{ inputs.codecov_token }} | ||
verbose: true | ||
gcov: true | ||
- name: Create codeclimate coverage | ||
run: |- | ||
lcov --directory . --capture --output-file lcov.info | ||
lcov --remove lcov.info -o lcov.info '/usr/*' | ||
sed "s=^SF:$PWD/=SF:=g" -i lcov.info # relative pats are required by codeclimate | ||
mkdir -p /tmp/codeclimate | ||
cc-test-reporter format-coverage -t lcov -o /tmp/codeclimate/${{ inputs.flags }}.json lcov.info | ||
shell: bash | ||
- uses: actions/upload-artifact@v3.1.1 | ||
with: | ||
path: "/tmp/codeclimate/*.json" | ||
name: codeclimate |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess we want to revert the changes to this file before merge, right?