Contributing Source Code to AOM

Revised 2016-04-5

This document will help guide you through contributing code to the AOMedia Project.


Contents


In general, most of the on-going work on AOM concerns speed and quality improvement. A developer typically follows the following workflow:

  1. Work on the source code.
  2. Recompile the aomenc encoder binary.
  3. Perform a video encode job with the new binary.
  4. Evaluate the results, comparing to an identical encode done with a prior version of aomenc.

Once meaningful improvement is achieved, the developer submits their code revisions to AOM.

Below we describe in more detail how the core AOM developers evaluate their own work, so that new contributors can replicate the process and have their code contributions approved and merged more easily.

Required Packages

  • A Bash shell
  • Python
  • A modern web browser

Download Test Clips

  1. Create a directory to hold the clips (e.g., ~/derf_cif on Linux or c:/derf_cif on Windows).
  2. Visit http://media.xiph.org/video/derf/ with your web browser.
  3. Download each of the the CIF clips found there, in YUV4MPEG (y4m) format, to the directory you created.

Encoding Only a Few Test Clips

Normally, you won’t want to encode the entire set of test clips, which can be very time-consuming. Instead, consider symlinking only the clips you want to use from their storage directory (e.g. ~/derf_cif) to a working directory (e.g. ~/derf_working), and running your tests there.

Download Test Scripts

Test scripts (and this document) are available in the contributor-guide Git project:

$ git clone https://aomedia.googlesource.com/contributor-guide
$ cd contributor-guide

Symlink or physically copy the following scripts to somewhere in your $PATH:

  • be
  • run_tests
  • visual_metrics.py

Download, Build and Test AOM

  1. To get started, ensure that your system satisfies the build prerequisites.
  2. Next, clone the AOM repository. After cloning, you’ll be working in the master branch by default.
    
    $ git clone https://aomedia.googlesource.com/aom
    $ cd aom
    
  3. Configure and build AOM. Be sure to run the configure script with --enable-internal-stats, which causes the encoder (aomenc) to output statistical data as well as a video file. For example:
    
    $ ./configure --enable-internal-stats --enable-debug --disable-install-docs --log=yes
    $ make
    

    To learn more about configuration options, do:

    $ ./configure --help
    

    There are several ways to “install” (make install, etc.), so we’ll leave that up to you. Just ensure that aomenc (aomenc) is executable and in your $PATH.

  4. Run some encodes using the provided scripts. Example:
    $ cd <test clip directory>
    $ run_tests 200 500 50 baseline
    

    In this example, run_tests will encode each y4m file in the current directory at bitrates of 200, 250, 300, 350, 400, 450 and 500 kbps (start at 200, end at 500, 50 kbps increments). It will create a directory baseline and write its output there.

    Output consists of webm format video files and stt files containing statistics for each encode. If the test encodes are 2-pass (default), an fpf file (first-pass data file) is also written for each input file, and will be reused on subsequent 2-pass encodes if not deleted.

  5. Make your changes to AOM, rebuild, and re-run the tests using a different output directory name. Example:
    $ run_tests 200 500 50 mytweak
    

    Alternatively, change the parameters in the batch encoder script (be) that’s called by run_tests. be in turn calls aomenc.

  6. When the tests are finished, run visual_metrics.py to generate an HTML-format report that compares mytweak with baseline.
    $ visual_metrics.py metrics_template.html "*stt" baseline mytweak > mytweak.html
    
  7. View the report by opening mytweak.html in a web browser.

Submit Your Patches

We manage the submission of patches using the Gerrit code review tool. This tool implements a workflow on top of the Git version control system to ensure that all changes get peer reviewed and tested prior to their distribution.

Browse to https://aomedia.googlesource.com/ and login with your account (Gmail credentials, for example). Next, follow the Generate Password link at the top of the page. You’ll be given instructions for creating a cookie to use with our Git repos.

Contributor Agreement

You will be required to execute a contributor agreement to ensure that the AOMedia Project has the right to distribute your changes.

Testing your code

The aom code base includes a set of unit and system tests that can be built and run locally as follows:

make test -j32

This downloads any required test material (raw video files etc) and runs the tests we have in parallel on 32 cores.

The tests all use googletest (read more here: https://github.com/google/googletest)

These tests and many more (e.g. asan, tsan, valgrind) will run through jenkins instances (when we hook this up) on upload to gerrit.

Coding style

We are using Google C Coding Style defined here https://google.github.io/styleguide/cppguide.html.

To facilitate style-guide compliance we’ve create a .clang-format file in the root directory https://chromium.googlesource.com/webm/aom/+/master/.clang-format

Download clang-format version 3.7.1 here : http://llvm.org/releases/download.html and insure that your executable path includes the path to git-clang-format.

Documentation here: http://llvm.org/releases/3.7.0/tools/clang/docs/ClangFormat.html

The following table shows some sample commands you can use to auto reformatting the code:

Command Description
git clang-format Apply clang-format to all staged changes.
git clang-format -f Clang format all staged and unstaged changes.
git clang-format -f -p Clang format all staged and unstaged changes interactively.

Jenkins runs a linter on code pushed to gerrit and will create a review with any style violations.

Clang-format has also reportedly been integrated into VisualStudio.

commit-msg hook

Gerrit requires that each submission include a unique Change-Id. You can assign one manually using git commit --amend, but it’s easier to automate it with the commit-msg hook provided by Gerrit.

Copy commit-msg to the .git/hooks directory of your local repo. Example:

$ curl -Lo aom/.git/hooks/commit-msg https://chromium-review.googlesource.com/tools/hooks/commit-msg

Next, ensure that the downloaded commit-msg script is executable:

$ chmod u+x aom/.git/hooks/commit-msg

See the Gerrit documentation for more information.

Upload Your Change

The command line to upload your patch will be of the form:

$ git push https://aomedia-review.googlesource.com/aom HEAD:refs/for/master

Incorporating Comments

If you previously uploaded a change to Gerrit and the Approver has asked for changes, follow these steps:

  • Edit the files to make the changes the reviewer has requested.
  • Recommit your edits using the --amend flag, for example:
    $ git commit -a --amend
  • Use the same git push command as above to upload to Gerrit again for another review cycle.

In general, you should not rebase your changes when doing updates in response to review. Doing so can make it harder to follow the evolution of your change in the diff view.

If your commit does not include a ‘Change-Id:’ line in its commit message (see commit-msg-hook, above), you’ll have to add one. Do so by running git commit --amend to edit the commit message, and then create a ‘Change-Id: I …’ by using the ID number of the change you’re trying to update, which can be found using the web UI.

Submitting Your Change

Once your change has been Approved and Verified, you can “submit” it through the Gerrit UI. This will usually automatically merge your change into the branch you specified, creating a new merge commit in Git if neccessary. Sometimes this can’t be done automatically. If you run into this problem, you must rebase or merge your changes manually. We strongly prefer that you rebase your changes instead of merging them. To do so, run:

$ git fetch
$ git rebase origin/branchname

If you get any conflicts, resolve them as you normally would with Git. When you’re done, reupload your change.

Viewing the Status of Uploaded Changes

To check the status of a change that you uploaded, open Gerrit, sign in, and click My > Changes.

Report Bugs

You can report bugs in either the code or the specification document using the AOMedia issue tracker.