Large File Error dont knpw what to do

[Chidifinance]

Select Topic Area

Question

Body

I've trying to push a branch to my repo. It keeps timing out and I keep getting the below error. I've already deleted the folders from my local repo, so not sure what else I should do

Please help

Error MSG:
[2024-09-16T20:07:06.524Z] remote: warning: File TEST_FILES/TEST/WFA/2024-23-08/UNANCHORED/MAStrat_AAPL_PFE_AMD_MSFT_GOOGL_AMZN_lookback_252_val_126_warmup_300.pkl is 97.54 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File TEST_FILES/TEST/WFA/2024-21-08/AAPL_PFE_AMD_MSFT_GOOGL_AMZN_lookback_1008_val_252_warmup_300.pkl is 99.41 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File TEST_FILES/TEST/WFA/2024-21-08/AAPL_PFE_AMD_MSFT_GOOGL_AMZN_lookback_504_val_252_warmup_300.pkl is 74.12 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File TEST_FILES/TEST/WFA/2024-21-08/AAPL_PFE_AMD_MSFT_GOOGL_AMZN_lookback_252_val_252_warmup_300.pkl is 59.66 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File TEST_FILES/TEST/WFA/2024-22-08/UNANCHORED/MAStrat_AAPL_PFE_AMD_MSFT_GOOGL_AMZN_lookback_252_val_252_warmup_300.pkl is 52.29 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File TEST_FILES/TEST/WFA/2024-22-08/UNANCHORED/MAStrat_AAPL_PFE_AMD_MSFT_GOOGL_AMZN_lookback_504_val_252_warmup_300.pkl is 65.21 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File TEST_FILES/TEST/WFA/2024-22-08/UNANCHORED/MAStrat_AAPL_PFE_AMD_MSFT_GOOGL_AMZN_lookback_1008_val_252_warmup_300.pkl is 87.90 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File TEST_FILES/TEST/Module Tests/raw_code/logs/genData_Vector.py.log.2024-09-08 is 63.99 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File TEST_FILES/TEST/WFA/2024-21-08/AAPL_PFE_AMD_MSFT_GOOGL_AMZN_lookback_756_val_252_warmup_300.pkl is 87.21 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File TEST_FILES/TEST/WFA/2024-22-08/UNANCHORED/MAStrat_AAPL_PFE_AMD_MSFT_GOOGL_AMZN_lookback_756_val_252_warmup_300.pkl is 77.13 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: error: Trace: a968c9a3600c490030209a78142953f71cea8564a8801504872d477df43e106d
remote: error: See https://gh.io/lfs for more information.
remote: error: File TEST_FILES/TEST/WFA/2024-21-08/AAPL_PFE_AMD_MSFT_GOOGL_AMZN_lookback_756_val_126_warmup_300.pkl is 169.12 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: File TEST_FILES/TEST/WFA/2024-21-08/AAPL_PFE_AMD_MSFT_GOOGL_AMZN_lookback_504_val_126_warmup_300.pkl is 141.40 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: File TEST_FILES/TEST/WFA/2024-21-08/AAPL_PFE_AMD_MSFT_GOOGL_AMZN_lookback_1008_val_126_warmup_300.pkl is 194.41 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: File TEST_FILES/TEST/WFA/2024-21-08/AAPL_PFE_AMD_MSFT_GOOGL_AMZN_lookback_252_val_126_warmup_300.pkl is 111.28 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: File TEST_FILES/TEST/WFA/2024-23-08/UNANCHORED/MAStrat_AAPL_PFE_AMD_MSFT_GOOGL_AMZN_lookback_504_val_126_warmup_300.pkl is 124.54 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: File TEST_FILES/TEST/Module Tests/raw_code/logs/genData_Vector.py.log.2024-09-13 is 115.36 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: GH001: Large files detected. You may want to try Git Large File Storage - https://git-lfs.github.com.
To https://github.com/Zino-ctrlZ/stop-loss-script.git
! [remote rejected] rebase_issue -> rebase_issue (pre-receive hook declined)
error: failed to push some refs to 'https://github.com/Zino-ctrlZ/stop-loss-script.git'

[barishaxxer]

You see this error because Github doesnt allow you to push files exceeding 100mb.to ease managin big files you should use something known as LFS.
https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-git-large-file-storage
https://learn.gitkraken.com/courses/take/git-foundations/lessons/38725159-what-is-git-lfs

[Chidifinance]

Is it possible to delete these files? I dunno how to use or switch to this lfs stuff.

[barishaxxer]

Sure.You need to remove this files from also git history so you can free space.if you dont have git filter-repo first install it by:

pip install git-filter-repo

then remove large files using:

git filter-repo --path large_file_1 --path large_file_2 --invert-paths

--invert-paths will remove the files instead of preserving them.
or just create a .gitignore and add file names that you dont want to push

Edit: you may have to force the push after this operations if something failes again after processing commands that i have mentioned

git push origin main -f

and hopefully this should solve your problem <3

[Sin-cy]

Someone actually solved this issue by running this command. Its the issue with pushing size. You can try running in your shell
git config --global http.postBuffer 524288000

[Chidifinance]

Tried this already and doesn't work :(

[Sin-cy]

Hmmmm not sure if this will help either. Did the errors show up the same as the first one ?
Maybe you can try getting the commit hash of the latest commit you made ready to push it specifically by doing
git log --oneline

to grab the 7 digit character from ur latest commit ( hasd992 )
and run the command below with ur hash

git push origin hasd992:main