OMM BPC cross-analysis

OMM sections

PDOC: Project documentation
STD: Use of Established and Widespread Standards
QTP: Quality of the Testing Process
LCS: Licenses and IP
ENV: Software Environment
DFCT: Commits and bug reports
MST: Maintainability and Stability
CM: Configuration/Version Management
PP: Project Planning
REQM: Requirements Management
RDMP: Roadmap
STK: Stakeholders
CLD: Cloud Deployment

Cross-analysis

	OMM	MISS	PDOC	STD	QTP	LCS	ENV	DFCT	MST	CM	REQM	RDMP	STK	CLD
Best Practices Criteria
MISS				X							X	X	X	X
Basics	Project website	X
	Basic project website content	X
	FLOSS license		X			X
	Documentation		X
	Other: HTTPS Web discussions English language
Change control	Public version-controlled source repository									X
	Version numbering	Semantic versioning Usage of tags								X
	Release notes (ChangeLog)	List of pbulicly known fixed vulnerabilities								X
Reporting	Bug reporting process							X
	Vulnerability reporting process	X
Quality	Working build system	X
	Automated test suite						X
	New functionality testing	X
	Warning flags	X
Security	Secure development knowledge	X
	Good cryptographic practices	X
	Secured delivery mechanism	X
	Publicly known vulnerabilities fixed	X
	Other security issues	X
Analysis	Static code analysis	The OW2 mature level requires static analysis but the criteria is not present in OMM itself
	Dynamic analysis	X
Future	Installation
	Reproducible build	X
	Crypto network	X
	Crypto certificate	X
	Hardening	X

OMM

Project Documentation

Purpose: Develop and maintain project documentation, making it readily accessible to the community

Documentation maintenance
Asset documentation	Documentation related to assets need to be provided and properly maintained. Access needs to be provided for the community and relevant stakeholders.
PDOC-1.1	Does the project create a development documentation?	Check the availability of requirements specification Check the availability of high level design / product architecture Check the availability of detailed design\|Check the availability of technical documentation (e.g. for use in debugging) Check the availability of workflow guidelines (for checking, testing...)
PDOC-1.2	Does the project create user documentation?	Check the availability of a user's guide Check the availability of FAQ documents Check the availability of training material on how to use the product (Multimedia rich material)
PDOC-1.3	Does the project create generic documentation?	Check the availability of roadmap documents Check the availability of a list of all documents available Check the availability of books describing the project (number, language...) Check other web based resources in the internet by third party
PDOC-2.1	Does the project provide a documentation system?	Check the availability of a documentation indexing feature (searching keywords for example) Check the availability of a documentation management system (ex. Epiware, KnowledgeTree, Alfresco, Plone, OpenDocMan...)
PDOC-2.2	Does the project maintain the documentation based on the collected user feedback?	Check for evidence of systematic evaluation of user feedback and maintenance Check the availability of a set of wiki pages, forums, or mailing lists focused on project's documentation
Documentation improvement	Quality and scope of the project and asset documentation should be continuously improved.
PDOC-3.1	Does the project have a documentation roadmap?	Check the availability of documentation aspects in the roadmap of the project\|Ensure that the documentation is appropriate for the version of the product
PDOC-3.2	Does the project improve documentation through support of several natural languages?	Check the availability of the documentation in different languages
PDOC-3.3	Does the project improve intrinsic quality of documentation?	Check if the documentation is better than before
PDOC-3.4	Improve documents based on feedback and on evaluation	Check “Change History” of documents for improvements based on feedback from users or evaluation

Use of Established and Widespread Standards

Purpose: Promote the use and implementation of Open Standards for product and process

Use of open standards
STD-1.1	Does the project implement widely accepted open product standards?	Look for implementation of open standards in the area of: Networking (TCP/IP, SSL, SMTP, MIME, IMAP, LDAP, WWW-HTTP...) Web content (HTML...) Email (SMTP, Email, WWW-MIME...) Doc Exchange (XML …) Graphics (PNG …) Window System (X Window …) Audio (Ogg, Vorbis …) Office documents (OpenDocument …) Web services (UDDI, SOAP…)
STD-1.2	Does the project implements standards certified by FLOSS-supporting certification entities?	The project implements standards proposed by the following certification entities: IETF, IEEE, OASIS, W3C, Free Standards Group, other FLOSS supporting entities
STD-1.3	Does the project identify clearly the standards relevant to the product?
Strategic independence	To ensure the sustained viability of a project and to improve interoperability with other projects, measures should be taken to make assets and processes independent of proprietary solutions.
STD-2.1	Does the project ensure independence from specific technologies?	Independence from proprietary standards Independence from proprietary development tools Independence from proprietary modules and software components Independence from proprietary data formats

Quality of the Testing Process|Purpose: Provide and implement a high quality testing process

Quality of the test plan
QTP-1.1	Does the project ensure that the test plan covers functional testing?
QTP-1.2	Does the project ensure that the test plan covers non-functional testing (as demanded by the project; see below)?	Scalability Maintainability Usability Performance Security Stability Testability internationalization/localization Other non-functional test as demanded by the specific project
QTP-1.3	Does the project ensure that the test plan covers different testing approaches?	unit testing integration testing includes system testing system integration testing
QTP-1.4	Does the project define test cases and testing criteria?	user requirements architecture requirements technical requirements\| testing process requirements
Managed testing process
QTP-2.1	Does the project conduct tests in regular intervals according to plan?	Conduct different tests regularly Align the testing process with project's roadmap
QTP-2.2	Does the project ensure that the test results are documented and available?	Past tests are reported, explained and results are presented The project provides detailed test results Feedback on past tests is obtained and is available for review
Testing process improvement	The following objectives aim on helping projects to continuously improve their test processes.
QTP-3.1	Does the project include test cases, test results and consider comments from stakeholders of the FLOSS project?	from developers, occasional contributors, etc. from users from integrators bugs identified (number, type, and severity) functional requests\|performance actions Check that a link between test cases and change requests /comments exists

Licenses, Copyright and IP Management

Purpose: Ensure the project has appropriate intellectual property policy, selects appropriate licenses and copyrights and manage them satisfactorily. The text in this section does not constitute legal advice. It is important to make clear what licenses we are talking about or more precisely what components need to be taken into account. It is not just the components "in the repository" (or "in the source folders or archives"). One has to look at libraries, and the components the software will be linked to.

Asset license management
LCS-1.1	Has the project evaluated available choices against licenses already used inside the FLOSS project, or the license strategy/policy inside the company when choosing its license?	For communities: Is the compatibility of the license of new components with the already used licenses checked regularly? (e.g is there a process defined to do this?) New licenses adopted or linked to the project are compatible with the one already used. For integrators policy/strategy: Has the project identified options for FLOSS licenses to use according to company strategy
LCS-1.2	Does the project clearly manage its licensing conditions?	The project clearly displays a license notice on its website.\|The complete license text is available with the product.\|Licensing conditions and their implications are unambiguously described. In particular for purposes of software integration and further linking to other products / components.\|All files have a license explicitly stated at each file (otherwise at file level it is difficult to decide whether a files does not have a license or if the license is inherited from the project). Number of files with no explicitly listed license/modules (evolution ratio).
Copyright management and management of intellectual property
LCS-2.1	Has the project selected appropriate copyright and IP policy?	Has the project identified a policy for code ownership (e.g. contributions may belong to their individual authors versus a single entity)? Is the copyright and IP policy chosen by the project compatible with the license of the project?
LCS-2.2	Does the project clearly manage its copyright and IP policy?	Copyright and IP policy available at the project level (website) Copyright statement is available at the project level\|Copyright statement is available in individual project files In the case the ownership of the code belongs to a well-defined entity, presence of a Contributor Agreement
Improved diffusion of FLOSS
LCS-3.1	Ensure that the product does not contain or use components that are not distributed under a FLOSS license (and the licenses of the components need to be compatible)
LCS-3.2	The license(s) is(are) endorsed by the OSI (http://www.opensource.org/licenses)

ENV: Environment

Purpose: Provide development resources such as operating system, language compilers, development tools, communication tools, ideally all tools should be FLOSS-compliant and interoperable

Provisioning of development resources and infrastructure
ENV-1.1	Does the project ensure availability of software tools and environment required for development?	The project has specialized mailing lists for users/developers The project has specialized forums for users/developers The project has specialized wiki pages for users/developers The project supports audio/video conferencing facilities The project has audio/video messages presenting key aspects of use and development The project has a list presenting and describing all the tools used inside the community for specific purposes The project supports the use of tools that are developed in order to improve the FLOSS development
ENV-1.2	Does the project ensure availability of methodologies required for development?	The project has workflows for checking, testing, contributing, etc Availability of technical documentation of the methodologies
ENV-1.3	Are there integrated management and communication tools used by the project (select the tool relevant to the project, see examples below)?	BlueFish\|Anjuta\|Glade\|GCC\|Kdevelop\|GDB\|KompoZer\|Eclipse\|The project is well integrated in important forges
Continuous maintenance of the project environment
ENV-2.1	Can users suggest modifications to the environment?	The users have the possibility of commenting the choice of tools The project takes into consideration comments from users (e.g. mailing list, email communication)
ENV-2.2	Does the project maintain the development environment corresponding to each maintained FLOSS version?	list of tools used for a version and the tools themselves
Encouragement to use FLOSS tools
ENV-3.1	Does the project gradually phase out commercial development resources and tools in favour of FLOSS equivalents?	The project has a clearly defined plan to phase out commercial development resources and tools and to introduce FLOSS components Transition to FLOSS from proprietary tools is documented and presented

Maintenance of defects, commits, and other contributions

Purpose: Specifically analyze the activity related to source code commits and bug reports provided to the project

Asset maintenance
DFCT-1.1	Does the project implement good FLOSS maintenance practices?	The project has a clear maintenance policy\|Delegate authority to a group of developers for the maintenance process Separate maintenance work on older version of the product from the work on the current version of the product Provide separate mailing lists, forums, web pages for maintenance
DFCT-1.2	Does the project maintain the most often used older versions of the product?	The number of old versions that are actively maintained is large The duration for which old versions of the product are maintained is large (up to two years)
Environment for contribution of bug reports and commits
DFCT-2.1	Does the project provide a standardized and well-documented contributing mechanism?	Corrections to source code and new code can be contributed easily\|Bug reports can be contributed easily\|Corrections to documentation can be contributed easily\|Suggestions for new features can be contributed easily\|The defect reporting system is well aligned with the development environment
Management of contributions, commits, and bug reports
DFCT-3.1	Does the project manage the defect reports?	The project processes the defects submitted at fixed intervals The project resolves defects based on nature of defect (based on severity of defect) The project updates defect report showing revision history
DFCT-3.2	Does the project create and maintain a data repository of closed defects?	source code related to the closed defects can be easily retrieved Bug reports can be easily retrieved Documentation contributions can be easily retrieved All contributions can be searched according to different criteria (e.g. topic, source code file name, author, distribution of the project, date of contribution, complete set of subsequent changes of the contribution)
Encouragement of defect reporting
DFCT-4.1	Does the community respond efficiently to defect reports?	Efficiency of response time related to resolving defects Efficiency of response related to source code inclusion into the new distribution
DFCT-4.2	Does the project offer specific recognition to users who report defects?	The project grant users voting privileges inside the community\|Users reporting defects are singled out in mailing lists, forums, etc.

Maintenance of non-functional properties

Purpose: Specify, develop and maintain non-functional requirements for the project (e.g. maintainability, stability)

Asset quality plan reflects non-functional requirements
MST-1.1	Does the project ensure that design and code are maintainable and stable (e.g. through best practices)?	The project conducts regular stability tests\|The project has a well defined / stable architecture Completeness of single modules and contributions are tested in the project\|Consistency of single modules and contributions are tested in the project Reliability of single modules and of the whole product are tested in the project The Maintainability index is calculated, presented and positive (lines-of-code measures, McCabe measures and Halstead complexity measures)
MST-1.2	Does the project assess the interoperability and compatibility of old and new versions of the product?
MST-1.3	Does the project conduct stability tests on different software and hardware systems?	Tests on different operating systems (Linux, Windows, MAC, Solaris, BSD etc) Tests using different compilers\|Tests using different hardware (different PC configurations, mobile devices, Macintosh computers, etc) Tests using different software configurations necessary for the functioning of the product
Asset maintenance management
MST-2.1	Does the project enforce good maintainability of the software?	Define goals for maintainability Delegate authority to a group of developers for the product maintenance\|Separate the maintenance of previous versions of the product from the work on the current version of the product Provide the tools for the maintenance process (separate mailing lists, forums, web pages) New additions and changes follow project internal standards (such as coding guidelines; workflow guidelines)
MST-2.2	Does the project maintain the most often used older versions of the product?	Number of old versions that are actively maintained > 0\|Duration for which old versions of the product are maintained adapted to user requirements

Configuration / Version Management|Purpose: Establish and maintain the integrity of the product and its releases

Release management efficiency
CM-1.1	Does the project identify configuration items?
CM-1.2	Does the project use a configuration / version management system (SVN, Git, etc.)?	Usage of a configuration management system Configuration management system access control procedures Change request database connected to the configuration management system
CM-1.3	Does the project manage releases?	Availability of releases\|Description of releases
Tracking and control of changes
CM-2.1	Does the project track change requests?
CM-2.2	Does the project control changes in configuration items?	Revision history of configuration items Archives of the releases
Integrity of asset releases
CM-3.1	Does the project manage configuration management records?	Revision history of configuration items Change log Annotation of the change requests Status of configuration items Document differences between releases

Cloud Deployment

Purpose: assess cloud deployment readiness throuh quality requirements that have to be met in order to ensure that a project delivers deployable assets (binaries, images, templates) within a cloud marketplace such as AppHub.

CLD-1.1	Does the project provide cloud templates or images that are ready for deployment?	The project shall provide templates or virtual images in one or several cloud deployment platforms, such as AppHub The project shall make sure that the templates or images contain installation instructions and release notes
CLD-1.2	Does the project provide deployable templates / images versions that are aligned with the project's releases?	Projects shall provide several versions of the deployable asset following the project release cycle (whether binaries or templates/images) Projects shall make sure that the release notes in one or several cloud deployment platforms such as AppHub log the improvements and differences between versions.
CLD-1.3	Does the project define a procedure for managing the deployable templates / images (e.g. as part of the configuration / release management)?	The project shall appoint a person in charge of producing and exposing the deployable templates / images in one or several cloud deployment platforms such as AppHub The project shall document the release of the deployable templates / images (internal documentation, or on the project website)
CLD-2.1	Does the project provide evidence that the deployable assets function correctly?	The project shall make sure that there is a description available associated to the project asset, that explains how to test whether the deployed asset works correctly The project shall provide an automated test suite on deployable assets.

Project Planning

Purpose: Estimate and plan project activities

Estimates
PP-1.1	Does the project define its scope?	Task descriptions\|Work package descriptions Work breakdown structure (WBS)
PP-1.2	Does the project define its release plan?
PP-1.3	Does the project estimate efforts?	Estimation rationale and assumptions for estimates are clear The project estimates efforts\|Inventory of skills needed and available Critical facilities/equipment list
Project plan
PP-2.1	Does the project establish the schedule or milestone plan?	Project schedules Schedule dependencies Process/workflow definitions and diagrams

Requirements Management|Purpose: Establish and maintain Requirements

Purpose: Establish and maintain Requirements

REQM-1	Does the project have a way to define relevant requirements?	The project identifies appropriate requirements providers (users, specifications, etc.) Criteria for evaluation and acceptance of requirements\|List of valid requirements
REQM-2	Does the project manage requirement changes?	Requirements status\|Requirements database Repository of requirement change decisions

Roadmap

Purpose: Create and maintain product roadmap

RDMP-1	Does the project define responsibility for the roadmap?	Define responsibility of developers involved in the definition and management of the roadmap
RDMP-2	Does the project efficiently manage its roadmap?	Details for different aspects of development (bugs, issues, features, releases etc) The roadmap is prepared for a period longer than one year, or the next two major versions / releases The roadmap contains a detailed description of new features The roadmap contains a schedule for new features and future releases The roadmap elements are well described indicating also their importance for the future version of the product
RDMP-3	Does the project define different types of releases?	Check for major or milestone releases Check for minor releases (bug fixes, release candidates, etc.)

Stakeholders|Purpose: Plan and manage stakeholder involvement

Stakeholders management planning
STK-1.1	Does the project identify its stakeholders (users, contributors, community)?
STK-1.2	Does the project share information with its stakeholders?	Ways to inform the stakeholders (website, mailing lists, newsletter, …)	Regular meetings of key stakeholders and dates for future meetings
STK-1.3	Does the project monitor the communication traffic inside the community?	Module authorship (number of modules owned by authors) Code leadership (percentage of total number of modules owned by author) Number of active developers turned into core developers\|Ratio users/developers/core-developers Ratio orphaned lines (without known active owner) Presence of authorship tree (for single modules) List of most active mailing list contributors The communication level inside the community is alive and diversified List of most active mailing list contributors Number of mailing lists subscribers (simple users) turned into developers (code contributing users) (evolution ratio)
Stakeholders involvement management
STK-2.1	Does the project encourage stakeholders to participate in the meetings where they are required?	Provide benefits to users that are contributing to the community (voting privileges, management privileges, source code modification privileges etc) Moderate mailing lists / forums\|Moderate bug/issue management systems Survey regularly users what is their satisfaction level with available communication channels\|Adapt communication channels according to new requirements and comments provided by users
STK-2.2	Does the project measure regularly communication inside the community?	Number of bug-tracking issues (evolution ratio) Number of bug-tracking issues unassigned (evolution ratio) Percentage of bug-tracking issues resolved (per week, per month etc) Number of writers in mailing lists (evolution ratio, classes: active, passive, quite active, very active) Number of mailing lists subscribers turned into developers (evolution ratio) Ratio of downloads this week over number of downloads in previous week Number of feature requests submitted by users/developers (evolution ratio) Number of bugs/issues submitted to the project (evolution ratio)\|Number of subscribers in the mailing lists (evolution ratio)
STK-2.3	Does the project measure the response rate inside different communication channels?	Provide strong reactivity based on roles assignment Provide strong reactivity in mailing lists Provide strong reactivity in bug solution Provide strong reactivity in issues consideration
Stakeholders involvement improvement
STK-3.1	Does the project measure the response level inside different communication channels and propose improvements?
STK-3.2	Does the project improve the management style inside the project?	Regularly assign roles related to different communication channels Regularly evaluate the quality of the communication inside the project
STK-3.3	Does the project improve the communication level inside the community?	Take actions to prevent flaming Define communication rules Assign responsibility related to abuse inside the communication channel inappropriate communication (flaming, etc.) leads to loss of privileges\|Require from users that want to start actively communications inside the community to explicitly agree with defined communication rules