[Proposal] End-user impacts and change log

[Joseph Reynolds]

This is a proposal to improve OpenBMC communication and get better 
release notes.

** Preamble **

Teams that use OpenBMC in other projects will not know OpenBMC as well 
as the development community, and they will not know what is in each 
release.  The main way for them to find out is by reading the release 
notes.  They will make decisions based on its contents.  That makes the 
release notes an important communication between the development 
community and its end-users.

We need to get better at communicating new and changed OpenBMC 
functions.  Doing so gives many benefits, including better release notes 
[1].

** The proposal **

1. Adopt a new practice to provide end-user impact statements when and 
where it make sense to do so.
2. Begin a project level change log to summarize user impacts.

** Details **

How do these changes help connect the folks who contribute to OpenBMC 
with the folks who use it?  The main points are:
1. Contributors write an "end-user impact" statement to notify users 
that something they might care about has changed.
2. Community members use end-user impact statements to create a change 
log which summarizes changes they consider significant.
3. Release planning summarizes the change log as an input to the release 
notes.

Items in detail:

1. Create end-user impact statements.

An "end-user impact" statement summarizes each change from the point of 
view of someone familiar with using the external interfaces.  It tells 
them what they need to know about what changed from their point of view, 
so they know what to test, how to adapt, etc.

When there is an impact, it should have details like:
- Is there an end-user impact (y/n)?
- Is this a new feature or function (where are the docs)?
- Is this a bug fix?
- What does the end-user need to know (one liner)?
These statements don't have to be perfect.  Any inkling you can provide 
that there is a user impact is helpful to the community.  The impact 
might change during the release as the design evolves.  Different 
contributors will see different impacts.  That's all normal and okay.

Are the following end-user impacts?
- Change to phosphor-webui that affects the browser display?  Answer 
"no" if it doesn't affect usage, or "Yes" if the user will likely notice 
or care.
- Bug fix in fan control? Answer depends.  Answer "yes" if the bug might 
start a fire.
- Refactor code?  No
- New Redfish API?  Yes; new features are positive user impacts.
- Adding new documentation?  No; although helpful, docs don't affect the 
end-user's use of the external interfaces
- BMCWeb retires old and accepts new TLS encryption algorithms?  Yes, it 
affects how you can connect to the BMC
- Changes to phosphor-rest or to D-Bus APIs?  No, despite their 
importance, these interfaces are technically internal to the BMC.  We 
might want to document user-impacts to these as well.
- Security fix?  Yes, probably, given today's climate
Feel free to come up with your own criteria, based on your estimation of 
an end-user.

Writing an "end-user impact" statement bridges two worlds.  Folks who 
work on the code may prefer to leave documentation for others or may not 
fully understand the impact of their change on BMC operations.  And 
nobody else may understand the change.  So who should write the end-user 
impact statement?  Anyone who understands the change and how it impacts 
end-users.  That could be developers, testers, or project managers. 
Multiple people can contribute over time.  And it should be okay to 
document an end-user impact in someone else's change.

It is important that all of the end-user impact statements be 
"findable".  Adding them as comments on selected GitHub issues is an 
obvious solution, and I suggest using the issue associated with the code 
change that caused the user impact. (We could use GitHub labels [8][9] 
to mark issues which have user impact.)

2. Create a change log from end-user impact statements.

Groups who care about end-user impacts include testers, security folks, 
release managers, and others.  They can scan the issues, read the impact 
statements, and use them to curate a change log [2] which lists all 
changes that might be interesting to users.  Each change log entry 
should have a brief description of the impact (from the user's point of 
view) with links back to the issue.  The change log could be an openbmc 
wiki page.  (And yes, some people's job includes curating change logs.)

We should have a project-wide change log which captures all impacts.  
(It is easy to ignore entries you don't care about.)  Anyone who cares 
about a user impact can add items to the change log.

How is the change log useful?  Testers will use the change log to help 
ensure test coverage and add facts about where the test cases are stored 
and whether the tests passed.  Release managers will use it to gauge 
progress toward goals.  Security folks will cover security changes.  
Technical writers, etc.  Each will contribute significant facts about 
their part of the process, and the log is a place to organize links to 
demos, etc.

3. Use the change log to make release notes.

The Release Planning work group [3] can refer to the change log as an 
input to the Release Notes [1][4].  For example, multiple changes can be 
merged, minor changes can be omitted, important changes can be 
highlighted, etc.  Having good release notes is a hallmark of a mature 
development process.

Let's be clear about this: Most users will ignore the release notes or 
use it only to answer specific questions about bugs or features.  Only a 
few will use it to learn more about new functions, changed functions 
they have to adapt to, how to review test efforts, what security fixes 
are included, etc.  They will make decisions based on its content.  For 
example, some groups may use it as part of a software security assurance 
process [5].

** Conclusion **

That's the proposal ... create end-user impact statements ... summarize 
them in a change log ... and massage that into release notes.  The real 
trick is to actually write end-user impact statements as the work is 
being done.  We have the test team [6] and the security work group [7] 
to help with that, along with pressure from groups who use OpenBMC in 
their projects.

- Joseph Reynolds

[1]: https://en.m.wikipedia.org/wiki/Release_notes
[2]: https://en.m.wikipedia.org/wiki/Changelog
[3]: https://github.com/openbmc/openbmc/wiki/Release-Planning
[4]: 
https://github.com/openbmc/docs/blob/master/release/release-notes.md
[5]: https://en.wikipedia.org/wiki/Software_security_assurance
[6]: https://github.com/openbmc/openbmc/wiki/Test-work-group
[7]: https://github.com/openbmc/openbmc/wiki/Security-working-group
[8]: https://help.github.com/en/articles/about-labels
[9]: https://github.com/openbmc/openbmc/labels