-
Notifications
You must be signed in to change notification settings - Fork 143
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix API errors in SWT due to tests #1299
Fix API errors in SWT due to tests #1299
Conversation
57e98b0
to
f2ec94f
Compare
binaries/org.eclipse.swt.win32.win32.x86_64/META-INF/MANIFEST.MF
Outdated
Show resolved
Hide resolved
f2ec94f
to
57b745f
Compare
- Move Win32AutoscaleTestBase to package org.eclipse.swt - Increase minor version in manifest - Add missing @SInCE tags in extending test classes Fixes eclipse-platform#1298
57b745f
to
cb7188b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are tests added to the production bundle in the first place?
They belong to org.eclipse.swt.tests.win32
We never host tests in production bundles in platform.
This has been discussed before and the approach has been OK`d. See: |
This doesn't mean the test code has to be in same package as production. Everyone can now code against this "API" (which is not). This discussion was about fragments but the code is inside main SWT bundle I'm not sure why the new tests are so different to every existing test that they have to be in production code. @HeikoKlare : can we please move the tests to test bundles? |
I don’t think the tests end up in the production bundle do they? Let’s be fully informed before we make conclusions about actions to be taken. In my picture they are in the fragment and I do recall discussing that. |
Beside this, even if the test code stays where it is today (which I hope will not happen), we should never add ANY "since" versions to it but make sure that API tooling ignores this code completely (which would happen automatically once the test code would be moved to tests). |
I'm talking about SWT source tree right now, API errors and a major inconsistency introduced. I don't see final classes in the main SWT bundle, but that means, everyone with SWT projects imported in the workspace doesn't see same behavior of the code & tests as we see it at the runtime tests, same for API problems. This will be a major issue and time waste in the future if we start doing that, we have enough time spent on debugging SWT API problems in the past and I don't want anyone time be wasted here again. |
I am not sure if i can update here but still please let me know if this makes it more meaningful if updated in a different task. Thanks. Inspite of applying this patch on the latest SWT level i am still facing the below problems. Please let me know i missed out any steps for the same to be avoided or this changeset needs more addition with respect to the below errors as well. |
@iloveeclipse Can you please elaborate on that? I don't understand what you mean by "same behavior". |
If a test fails or code works in the workspace it doesn't guarantee it will fail on Jenkins and vice versa. E.g. add reference to any of the test classes to any SWT production class and it may be green locally in the workspace but I hope it will fail on Jenkins. if it will not fail on Jenkins, we have even a bigger issue. |
If the folder is correctly marked as a test source folder then no one will ever be able to reference anything here in regular/production code (otherwise its a bug in JDT)... that APi tools complains in the IDE might be a bug in PDE API tools that should not consider code located in folders not added in build.properties. |
Just curious, but why "hope will not happen" this is common practice in the software industry, its just Eclipse (PDE) that is special here and makes it hard to use/understand for newcomers that are used to the usual layout. |
This is not correct, as Eclipse is also part of the "software industry" and here it is not common practice. To speaking "maven" speak, if maven implies project layout for good reason, Eclipse implies separation of production and test code into different projects, also for good reasons, and that for every other project in the platform since ever.
It is not correct either, it is not about special PDE but about special change in question that broke consistent and easy to understand usual layout followed by any other Eclipse platform project. My problem is: all Eclipse Platform projects followed (till now) same common standards regarding tests. It was consistent, easy to understand and supported by tooling. Now we have an exception from this rule that is, in my opinion, not justified from technical point of view, not acceptable from platform project common practice point of view and shows first problem with the common tooling we use. This PR is just a "band-aid" for this main inconsistency, it makes no sense as the way it fixes the problem is wrong (there should be no need for the "API since" flags on non API code) and I'm against adding that and in favour of fixing original problem to avoid further problems arising from this band aid and the inconsistency. |
The justification from the technical point of view was provided in eclipse-platform/.github#203: it wasn't possible to test the microarchitecture of the Windows port with the current approach without elevating the classes that we wanted to test to a public API.
As @laeubi said in #1299 (comment)
Is a different subject. I could close this PR if this is the wrong fix for the errors in the API verification, but the tests should remain where they are. BTW I tried what you said about referencing the test class in production code and it is not possible (compilation fails, even for Linux) So, summarizing:
WDYT? |
Maybe not for Eclipse Platform, but anyways I don't have stated any thing that is is the only practice. Beside that it does not help to complain anyways, if no one would want/use it such feature would simply not exits.
Maven does not imply anything but is driven by convention over configuration, that means it was agreed that it is useful for a larger scale of users, still it does not forbids anything different used, it is just more work to configure.
Eclipse does not imply anything, also here you can do use different layouts, it is just that PDE/Tycho where limited in the past to a specific workaround that was claimed "best practice" because there was no other practices possible. But just because we are not used to it does not mean its useless/dangerous/bad/... and it is/was one of the often asked questions in the past why it is not supported by PDE/Tycho to have such layout. And as you have already found out, the code is seperated and JDT even marks test folders with a different visual layout, and it is even more easier to match test+code while with a seperate project there is some chance to miss it (project not imported or not noticed as its is "far away" from the implementation, ...)
I think we can accept different viewpoints and people are working differently so I really don't like to enter another fruitless endless "war of opinions here" there is simply no right/wrong, the only thing is that it was for a long time not possible (or very hard to archive).
As mentioned, before it was simply not possible, as before it was not possible to use maven-deps and everyone "followed" that standard... things are changing, so maybe even if it is uncommon until now this might or might not change, but trying it out and maybe see some pitfalls that needs fixing is nothing I would worry about.
I think we should verify that everything is setup correctly that is:
If that is the case I would expect the tooling to work and test code is neither shipped nor useable from "production" (even though some argue that test-artifacts are valuable deliveries for a product). If any issues are seen we should identify and fix them so whoever wants to use that (let it be platform or not) do not run into these as well. If Platform should or not should use it is a completely different topic (for me) as whatever we do someone will always be unhappy... so lets focus first on the technical aspects. |
Discussions seem to have stalled. It seems to me also that anything in a source folder marked as a test should also not be considered API by API tooling: But again, this requires some fix in the API tools. Perhaps I can find time to have a look, but that's only sensible if this conversation draws to a conclusion... |
@merks I know one needs to decide where to spend time, but
is completely independent (and at least deserves an issue at PDE), from:
I just would think it makes it easier to fix (and maybe helps with the discussion if solved) if we have a case to analyze now. |
FYI, I can see this in the .api_description of the most recent 4.33 I-build:
So I see no descriptions for the Test files. I assume that's a good thing. |
Sorry for being late in this discussion. I completely agree with everything written by @laeubi and @merks.
This absolutely makes sense also from my point of view, and I think this is also the correct way of dealing with the SWT API errors. @merks, are your experiments in PDE worth to be provided as a PR? Otherwise I (as the one having introduced this kind of tests) could also try to find some time to have a look into an improvement of the API tools. In my opinion, the tests are placed exactly where they need to be placed. I appreciate any pointers to alternative (technical) solutions, as currently I am not aware of any. The only "alternatives" I have seen so far (in platform code in general) is making things part of the bundle's API just for testing purposes (by providing packages or adding public methods only for testing) or by using reflection, but I consider none of them appropriate.
Just for completeness: the difference of these tests in comparison to all existing tests is that they are the first unit tests (i.e., not integration or system tests) for win32-specific SWT code. |
Definitely I will create a PR ASAP; I'm in Montreal until Thursday. I'll see what I can do quickly... |
I created the PR to improve/fix the API tools. Are any of the changes proposed here appropriate or should this PR be closed? |
From my understanding, changes should be obsolete when API tooling behavior is improved. But probably @fedejeanne should respond/decide on that. One additional thought: if I am not mistaken, using JUnit 5 instead of JUnit 4 would allow to reduce the test class visibility to package protected. This should also solve the API tools problem, but it would even avoid other potential tooling problems, which might lead to the test classes erroneously being considered as part of the API. And of course, keeping visibilities as low as possible is usually a good thing. So maybe that could be a follow-up task and a best practice for unit tests in production bundles? |
Oh wait, package with the public base test classes is internal so those can be public without being API. If the tests that extend them are all package protected, there would be no API warnings as is now because those things are necessary not visible downstream: That seems like a reasonable solution if the tests then still work. |
Exactly. It should be possible make each class either package protected or have it placed in a package that is not part of public API anyway. From a conceptual point of view, I do not see any other potential visibility-related constraints. Note that any potential restriction due to subclassing is something that can and should usually be avoided, as often there is no need for subclassing in tests (in particular when using JUnit 5) and there are very good reasons to avoid inheritance when writing tests in general (since JUnit 3 times are long over). I remember disentangling the |
I will hold off on merging the API tools changes until we establish there is no better way to simply avoid the problem via class visibility and/or internal packages. There's no point in complicating those tools or adding performance overhead to those tools if the problem can simply be avoided entirely. Thanks for your constructive suggestions! |
Maybe it should be fixed like this? |
Unfortunately, only reducing test class visibility but sticking to JUnit 4 does not seem to be possible (see #1303 (review)). Let's test a migration of the tests to JUnit 5: #1305 |
Maybe you can try removing the exported packages from the MANIFEST.MF. Tests are not API (PMC decision a while ago), therefore would be OK. Unless of course other tests need the exports. |
Better go with #1305 |
@since
tags in extending test classesFixes #1298