You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
tensorflow-allreduce/tensorflow/core/kernels/deep_conv2d.cc: ln 74,
static int64 GetDirectConvCost(int filter_rows, int filter_cols, int in_depth,
int out_depth, int out_rows, int out_cols) {
return filter_rows * filter_cols * in_depth * out_depth * out_rows * out_cols;
}
Can lead to integer overflow and weird results
I think, it should be smth like that
return (int64)filter_rows * (int64)filter_cols * (int64)in_depth * (int64)out_depth * (int64)out_rows * (int64)out_cols;
The text was updated successfully, but these errors were encountered:
tensorflow-allreduce/tensorflow/core/kernels/deep_conv2d.cc: ln 74,
static int64 GetDirectConvCost(int filter_rows, int filter_cols, int in_depth,
int out_depth, int out_rows, int out_cols) {
return filter_rows * filter_cols * in_depth * out_depth * out_rows * out_cols;
}
Can lead to integer overflow and weird results
I think, it should be smth like that
return (int64)filter_rows * (int64)filter_cols * (int64)in_depth * (int64)out_depth * (int64)out_rows * (int64)out_cols;
The text was updated successfully, but these errors were encountered: