Continuous Integration

[rgleason]

Revised: Advanced Testing Tips

Your suggestions for improvement will be appreciated.

[Gilletarom]

Originally Posted by rgleason

Revised: Advanced Testing Tips

Your suggestions for improvement will be appreciated.

Who is this question addressed to?
- Developers?
- To those who follow OpenCPN developments closely and who regularly beta-test?
- To all who access the Cruisers Forum?

[rgleason]

- Developers?
- To those who follow OpenCPN developments closely and who regularly beta-test?

[transmitterdan]

Rick,

IMO Beta testing is not appropriate for CI auto generated builds. Beta testing can only begin when the development team is sure there are no obvious bugs such as null pointer deref or other errors that can be caught through testing inside a debug environment. Beta testing begins when the development team wishes it. Developers will quickly tire of hearing about bugs they already know about. They also get fatigued by "testers" demanding changes to the UI or some other trivia whilst they are actively creating big new features or internal optimizations. Testers get upset when their "bug reports" go ignored. Continuous integration does not mean continuous beta testing. I think this is a move in the wrong direction and will likely alienate some testers and developers.

[Hakan]

TDan..
Very well expressed!
For whatever engineering alike work there must be a test and check period after first "issue for test". But seldom before unless asked for by the issuer. Normal engineering practice, I'd say.
Your note is much more important on a open community like OCPN. Otherwise work issues tend to be stored out of sight and we risk two or more are working on the same thing clueless of each others. Open cooperation and problem solving communication is a better way.

[rgleason]

TDan

Good points! We want to help the programmers, not bother them. Proposed clarification.

Alpha Testing
Done early on by skilled testers at the specific request of programmers who are actively creating big new features or internal optimizations. These skilled testers are asked by the programmers to help achieve the main goals of the program to get it ready for Beta Testing, using a "Debug" environment. They do not "Bug" the programmers with:
1. Additional feature requests.
2. UI changes.
3. Known Bugs (the programmers already know about)
4. Other trivia.
They debug the program using their software tools and report the actions and error codes using MSVC++ call stack, or Linux debug tools.
Beta testing can only begin when the development team is sure there are no obvious bugs such as null pointer deref or other errors that can be caught through testing inside a debug environment. Beta testing begins when the development team wishes it.

Beta Testing
Beta Testing is conducted in a Real Time environment (not using Debug) by volunteer testers, customers or end users when the development team determines the software is ready. Beta testing is not appropriate for CI auto generated builds. - (really? why not?)

CI auto generated builds are only good for "tasting the soup" as Dave says. The programmers know the software is not ready, so refrain from making any comments! Use the debug tools!

Opensource Cooperation
The notes above are much more important in an open community like OpenCPN. Otherwise work issues tend to be stored out of sight and we risk two or more are working on the same thing clueless of each others. Open cooperation and problem solving communication is a better way.

Observation
There is a valuable feedback loop missing here. When do users have an opportunity to:
1. Comment on difficult or impossible UI problems.
2. Make Feature Requests.

[rgleason]

Changes have been made here based on various comments here and elsewhere.
https://opencpn.org/wiki/dokuwiki/do...us_integration

I reversed the order, so perhaps it should be re titled? Other changes?

[rgleason]

Now it is separated into two pages:

Community

Advanced Debug Tips