-
Notifications
You must be signed in to change notification settings - Fork 667
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Applying autotest module on existing complex operators #10284
Merged
L-Xiafeng
merged 38 commits into
Oneflow-Inc:master
from
MarioLulab:luqi/autotest_on_complex
Jun 26, 2023
Merged
Applying autotest module on existing complex operators #10284
L-Xiafeng
merged 38 commits into
Oneflow-Inc:master
from
MarioLulab:luqi/autotest_on_complex
Jun 26, 2023
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…uqi/autotest_on_complex
MarioLulab
requested review from
BBuf,
liujuncheng,
hjchen2 and
chengtbf
as code owners
May 30, 2023 08:10
This was referenced May 30, 2023
levi131
pushed a commit
that referenced
this pull request
Jun 5, 2023
) ### Original requirements 现有的 conj kernel, real kernel, imag kernel 使用 KernelUtil 来进行组织,在 kernel 层面复数数据类型和实数数据类型的调用 conj 无法实现统一。故移除原有 KernelUtil 的实现,使用 ElementwiseUnary 的 primitive 进行实现 - [x] 新增 `kConj` 这一 UnaryOp,使用 ElementwiseUnary 的 Primitive 实现 conj,并移除原有 conj 的 KernelUtil - [x] 新增 `kReal` 和 `kRealGrad` UnaryOp,使用 ElementwiseUnary 的 Primitive 实现 real 和 real_grad,并移除原有 real 和 real_grad 的 KernelUtil - [x] 新增 `kImag` 和 `kImagGrad` UnaryOp,使用 ElementwiseUnary 的 Primitive 实现 imag 和 imag_grad,并移除原有 imag 和 imag_grad 的 KernelUtil ## 注意: 复数基础设施建设系列 pr: 1. 使用 primitive 来实现 conj, real 等常见复数算子: #10281 2. 将现有支持复数数据类型的算子测例迁移到 autotest 模块中,以使复数算子复用实数算子的测试用例:#10284 3. 继续拓展支持复数数据类型的算子,比如 matmul, sqrt, div 等:#10269 依赖关系: 本 pr 基于:[pr2](#10284) 和 [pr3](#10269) 的基础,请优先 merge 此 pr
skip complex fix ci
skip complex fix ci
skip complex fix ci
skip complex fix ci
skip complex fix ci
skip complex fix ci
fix test_case
format
format
levi131
approved these changes
Jun 16, 2023
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
L-Xiafeng
approved these changes
Jun 16, 2023
L-Xiafeng
requested review from
oneflow-ci-bot
and removed request for
oneflow-ci-bot
June 26, 2023 09:07
L-Xiafeng
added a commit
that referenced
this pull request
Jun 30, 2023
…10269) ### Original requirements AI for Science models FNO use a series of operators such as sqrt, mm, div, etc. for complex tensor. So we need to support complex tensor for these ops. #### Main Works **Support existing ops for complex tensor in this pr as below:** - broadcast_elementwise_binary | Op | complex type | Backend | Progress | |:-----:|:-------------:|:------:|:------:| | scalar_div | cp64, cp128 | CPU, CUDA | Done | | broadcast_div | cp64, cp128 | CPU, CUDA | Done | - broadcast_elementwise_unary | Op | complex type | Backend | Progress | |----------|:-------------:|:------:|:------:| | sqrt | cp64, cp128 | CPU, CUDA | Done | | negative | cp64, cp128 | CPU, CUDA | Done | - other exisiting operations | Op | complex type | Backend | Progress | |----------|:-------------:|:------:|:------:| | matmul | cp64, cp128 | CPU, CUDA | Done | | batch_matmul | cp64, cp128 | CPU, CUDA | Done | | broadcast_matmul | cp64, cp128 | CPU, CUDA | Done | | matrix_vector_product | cp64, cp128 | CPU, CUDA | Done | | reduce_sum_like | cp64, cp128 | CPU, CUDA | Done | ## 注意: 复数基础设施建设系列 pr: 1. 使用 primitive 来实现 conj, real 等常见复数算子: #10281 2. 将现有支持复数数据类型的算子测例迁移到 autotest 模块中,以使复数算子复用实数算子的测试用例:#10284 3. 继续拓展支持复数数据类型的算子,比如 matmul, sqrt, div 等:#10269 依赖关系: 本 pr 基于:[pr1](#10281) 和 [pr2](#10284) ,需要在 merge [pr1](#10281) 和 [pr2](#10284) 后,再 Merge 本 pr --------- Co-authored-by: levi131 <limaolin@oneflow.org> Co-authored-by: levi131 <83750468+levi131@users.noreply.github.com> Co-authored-by: lijunlin <70465472+L-Xiafeng@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Original requirements
Autotest: We found that the previous testing of operators supporting complex tensor was not complete. We decided to reuse the real tensor operator tests to ensure completeness. Since complex tensor tests are supported in the
autotest
module from pr (#10027) , in this pr we applied the autotest module to the tests of complex tensor operators already available in OneflowFix: In addition, the autograd rules for some previous operators of complex numbers might not conform to the convention of "Conjugate Wirtinger Derivative". We have fixed these bugs in this pr at the same time.
Main Works
Applying
autotest
module on existing operators that have already support complex tensor:Complex and Real Behave the Same Way
: means we don't need to add conjugate operation in op grad. Because regardless of whether the input data type involved in the operation is real or complex, the gradient result using the winterger derivative is the same as the real derivative rule,Grad Not Supported in OF
: means the grad of this op is not supported in oneflowbroadcast_elementwise_binary
broadcast_elementwise_unary
other exisiting operations
注意:
复数基础设施建设系列 pr:
依赖关系:
本 pr 基于:pr1,需要在 merge pr1 后,再 Merge 本 pr