Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File Downloader Submodule Major Refactor (Hi3Helper.Http) #563

Merged
merged 28 commits into from
Aug 31, 2024

Conversation

neon-nyan
Copy link
Member

@neon-nyan neon-nyan commented Aug 24, 2024

Main Goal

As per title, this PR contains some huge changes to our own File Downloader Submodule implementation that we use for Collapse called: Hi3Helper.Http.
The changes brings some new features to improve the reliability of the downloading process for certain parts, like Game Installation/Update, Game Repair, Cache Updates and Internal APIs. The changes that worth to mention are including:

  • [New] Bandwidth/Speed Limiter
    With this new feature, the user can now set the download speed limit.
  • [New] Download File Size Pre-allocation.
    Thanks to this feature, the download file would not need to have merged before the file can be used.
  • [Imp][Performance] Reduce unnecessary async thread overhead.
  • [Imp] Use delegates in replacement for event-based DownloadEvent to notify the download progress.
    This has benefits if you're performing download multiple files within a single Downloader instance for other purposes. By using delegates, you can subscribe the download progress to a specific file you wanted to track.
  • [Imp] Changing the language specification from C# 8.0 to C# 9.0.

However, this PR requires some changes in some parts, including:

  • Game Repair
    • Genshin Impact
    • Honkai Impact 3rd
    • Honkai: Star Rail
  • Cache Update
    • Honkai Impact 3rd
    • Honkai: Star Rail
  • Launcher API/Sprites Loader
    • HoYoPlay
    • Sophon
  • FallbackCDNUtil
  • Game Conversion Utils
  • InstallManagerBase
    • Genshin Impact
    • Honkai Impact 3rd
    • Honkai: Star Rail
    • Zenless Zone Zero
  • Launcher Updater

PR Status :

  • Overall Status : Completed
  • Commits : Completed
  • Synced to base (Collapse:main) : Not yet
  • Build status : OK
  • Crashing : No
  • Bug found caused by PR : -

Templates

Changelog Prefixes
  **[New]**
  **[Imp]**
  **[Fix]**
  **[Loc]**
  **[Doc]**

@neon-nyan neon-nyan added Enhancement New feature or request Area: Hi3Helper.Http Issue labeled for Hi3Helper.Http submodule issue releated Area: Runtime Issue labeled for runtime, Hi3Helper.Core and other libraries used by Collapse Area: Launcher Issue labeled for Launcher update mechanism labels Aug 24, 2024
@neon-nyan neon-nyan self-assigned this Aug 24, 2024
Copy link

github-actions bot commented Aug 24, 2024

Qodana for .NET

It seems all right 👌

No new problems were found according to the checks applied

💡 Qodana analysis was run in the pull request mode: only the changed files were checked
☁️ View the detailed Qodana report

Detected 102 dependencies

Third-party software list

This page lists the third-party software dependencies used in CollapseLauncher

Dependency Version Licenses
Clowd.Squirrel 2.11.1 MIT
CommunityToolkit.Common 8.3.0 MIT
CommunityToolkit.Mvvm 8.3.0 MIT
CommunityToolkit.WinUI.Animations 8.1.240821 MIT
CommunityToolkit.WinUI.Behaviors 8.1.240821 MIT
CommunityToolkit.WinUI.Controls.ImageCropper 8.1.240821 MIT
CommunityToolkit.WinUI.Controls.Primitives 8.1.240821 MIT
CommunityToolkit.WinUI.Controls.SettingsControls 8.1.240821 MIT
CommunityToolkit.WinUI.Controls.Sizers 8.1.240821 MIT
CommunityToolkit.WinUI.Converters 8.1.240821 MIT
CommunityToolkit.WinUI.Extensions 8.1.240821 MIT
CommunityToolkit.WinUI.Helpers 8.1.240821 MIT
CommunityToolkit.WinUI.Media 8.1.240821 MIT
CommunityToolkit.WinUI.Triggers 8.1.240821 MIT
GitInfo 3.3.5 MIT
Google.Protobuf.Tools 3.28.0 PROTOBUF
Google.Protobuf 3.28.0 BSD-3-Clause
H.GeneratedIcons.System.Drawing 2.1.3 MIT
H.NotifyIcon.WinUI 2.1.3 MIT
H.NotifyIcon 2.1.3 MIT
Hi3Helper.ZstdNet 1.6.2 BSD-3-Clause
HtmlAgilityPack 1.11.65 MIT
ImageEx 2.1.1 MIT
Markdig.Signed 0.37.0 BSD-2-Clause
Microsoft.CSharp 4.7.0 MIT
Microsoft.Graphics.Win2D 1.2.0 MS-ASP-NET-WEB-OPTIMIZATION
Microsoft.NET.ILLink.Tasks 8.0.5 MIT
Microsoft.NETCore.Platforms 8.0.0-preview.7.23375.6 MIT
Microsoft.NETCore.Targets 6.0.0-preview.4.21253.7 MIT
Microsoft.Web.WebView2 1.0.2651.64 BSD-3-Clause
BSD-MYLEX
Microsoft.Win32.Registry 5.0.0 MIT
Microsoft.Win32.SystemEvents 8.0.0 MIT
Microsoft.Windows.CsWinRT 2.1.1 MIT
Microsoft.Windows.SDK.BuildTools 10.0.26100.1 PROPRIETARY-LICENSE
Microsoft.WindowsAppSDK 1.6.240821007-preview2 MIT
MS-DXSDK-D3DX-9.29.952.3
Microsoft.Xaml.Behaviors.WinUI.Managed 2.0.9 MIT
PhotoSauce.MagicScaler 0.14.2 MIT
PhotoSauce.NativeCodecs.Libwebp 1.3.2-preview3 MIT
Roman-Numerals 2.0.1 MIT
SharpCompress 0.37.2 MIT
SharpHDiffPatch.Core 2.2.7 MIT
System.Collections.Concurrent 4.3.0 MIT
System.Collections 4.3.0 MIT
System.Diagnostics.Debug 4.3.0 MIT
System.Diagnostics.DiagnosticSource 4.3.0 MIT
System.Diagnostics.EventLog 8.0.0 MIT
System.Diagnostics.Tracing 4.3.0 MIT
System.Drawing.Common 8.0.8 MIT
System.Globalization.Calendars 4.3.0 MIT
System.Globalization.Extensions 4.3.0 MIT
System.Globalization 4.3.0 MIT
System.IO.FileSystem.Primitives 4.3.0 MIT
System.IO.FileSystem 4.3.0 MIT
System.IO.Hashing 8.0.0 MIT
System.IO 4.3.0 MIT
System.Linq 4.3.0 MIT
System.Net.Http 4.3.4 MIT
System.Net.Primitives 4.3.0 MIT
System.Reflection.Primitives 4.3.0 MIT
System.Reflection 4.3.0 MIT
System.Resources.ResourceManager 4.3.0 MIT
System.Runtime.Extensions 4.3.0 MIT
System.Runtime.Handles 4.3.0 MIT
System.Runtime.InteropServices 4.3.0 MIT
System.Runtime.Numerics 4.3.0 MIT
System.Runtime 4.3.1 MIT
System.Security.AccessControl 6.0.1 MIT
System.Security.Cryptography.Algorithms 4.3.0 MIT
System.Security.Cryptography.Cng 4.3.0 MIT
System.Security.Cryptography.Csp 4.3.0 MIT
System.Security.Cryptography.Encoding 4.3.0 MIT
System.Security.Cryptography.OpenSsl 4.3.0 MIT
System.Security.Cryptography.Primitives 4.3.0 MIT
System.Security.Cryptography.ProtectedData 8.0.0 MIT
System.Security.Cryptography.X509Certificates 4.3.0 MIT
System.Security.Principal.Windows 5.0.0 MIT
System.Text.Encoding.CodePages 8.0.0 MIT
System.Text.Encoding 4.3.0 MIT
System.Text.Encodings.Web 8.0.0 MIT
System.Text.Json 8.0.4 MIT
System.Text.RegularExpressions 4.3.1 MIT
System.Threading.Tasks.Extensions 4.5.4 MIT
System.Threading.Tasks 4.3.0 MIT
System.Threading 4.3.0 MIT
TaskScheduler 2.11.0 MIT
ThisAssembly.Constants 1.4.1 MIT
ZstdSharp.Port 0.8.1 MIT
runtime.debian.8-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2 MIT
runtime.fedora.23-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2 MIT
runtime.fedora.24-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2 MIT
runtime.native.System.Net.Http 4.3.0 MIT
runtime.native.System.Security.Cryptography.Apple 4.3.0 MIT
runtime.native.System.Security.Cryptography.OpenSsl 4.3.2 MIT
runtime.native.System 4.3.0 MIT
runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2 MIT
runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2 MIT
runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple 4.3.0 MIT
runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2 MIT
runtime.rhel.7-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2 MIT
runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2 MIT
runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2 MIT
runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.2 MIT
Contact Qodana team

Contact us at qodana-support@jetbrains.com

@bagusnl bagusnl marked this pull request as draft August 25, 2024 07:46
@neon-nyan neon-nyan marked this pull request as ready for review August 29, 2024 16:21
@neon-nyan neon-nyan requested a review from a team August 29, 2024 16:22
Copy link
Member

@bagusnl bagusnl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly codeqa and stuff to talk about but overall its all ok

@Cryotechnic Cryotechnic linked an issue Aug 30, 2024 that may be closed by this pull request
Copy link
Collaborator

@Cryotechnic Cryotechnic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, PR looks OK and works as intended (tested locally). We could cleanup the magic numbers that we use in the code, such as for specifying max thread count and such by declaring them as variables in the function but functionally it works as intended (mostly for debugging later, to make it easier). That being said, the following UI issue is encountered for Cache Repair in Hi3 GLB region. I don't have the other regions installed so cannot verify if it is region agnostic:

image

Furthermore, please make sure to fix the i18n issues before merging. Aside from that, great work!

Hi3Helper.Core/Lang/en_US.json Outdated Show resolved Hide resolved
Hi3Helper.Core/Lang/en_US.json Outdated Show resolved Hide resolved
Hi3Helper.Core/Lang/en_US.json Outdated Show resolved Hide resolved
Hi3Helper.Core/Lang/en_US.json Outdated Show resolved Hide resolved
Hi3Helper.Core/Lang/en_US.json Outdated Show resolved Hide resolved
Hi3Helper.Core/Lang/en_US.json Outdated Show resolved Hide resolved
Hi3Helper.Core/Lang/en_US.json Outdated Show resolved Hide resolved
Hi3Helper.Core/Lang/en_US.json Outdated Show resolved Hide resolved
bagusnl and others added 6 commits August 30, 2024 21:20
Co-authored-by: Kemal Setya Adhi <dev.kemalsetyaa@gmail.com>
Co-Authored-By: Ron Friedman <9833218+Cryotechnic@users.noreply.github.com>
@neon-nyan neon-nyan requested review from a team, Cryotechnic and bagusnl and removed request for a team August 30, 2024 19:12
@neon-nyan
Copy link
Member Author

Overall, PR looks OK and works as intended (tested locally). We could cleanup the magic numbers that we use in the code, such as for specifying max thread count and such by declaring them as variables in the function but functionally it works as intended (mostly for debugging later, to make it easier). That being said, the following UI issue is encountered for Cache Repair in Hi3 GLB region. I don't have the other regions installed so cannot verify if it is region agnostic:

image

Furthermore, please make sure to fix the i18n issues before merging. Aside from that, great work!

The issue with counts has a relation with the fetching mechanism to be enumerating the assets on-the-air rather than getting all the list of the assets and placed it into List<T>. Since it's using IEnumerable<T> to enumerate the assets OTA, getting the total count is not possible because technically IEnumerable<T> doesn't have a Count property.

Copy link
Member

@bagusnl bagusnl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All gud i aprob

Copy link
Collaborator

@Cryotechnic Cryotechnic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Send it \o/

@neon-nyan neon-nyan merged commit 71699b0 into main Aug 31, 2024
6 checks passed
@zakidze
Copy link

zakidze commented Sep 9, 2024

can you show some pictures on how to set the limit , also is it released yet on stable version????

and also please

i want to example duriang the day i donwlaod at 100kb /sec
at night i donwlaod at 1mb/sec
like can planify when speed get limited and when not

if this is not added

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Hi3Helper.Http Issue labeled for Hi3Helper.Http submodule issue releated Area: Launcher Issue labeled for Launcher update mechanism Area: Runtime Issue labeled for runtime, Hi3Helper.Core and other libraries used by Collapse Enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Request]: Download speed limit
4 participants