标题: [翻译]AFL 并行测试

分类: Fuzzing
创建: 2023-08-25 20:55
修改: 
链接: http://0x2531.tech/fuzzing/202308252055.txt

--------------------------------------------------------------------------------

=========================
Tips for parallel fuzzing
并行测试建议
=========================

  This document talks about synchronizing afl-fuzz jobs on a single machine
  or across a fleet of systems. See README for the general instruction manual.

  本文档讨论在单台机器或跨一组系统同步 afl-fuzz 作业。查看 README 文档了解一般说明。

1) Introduction
介绍
---------------

Every copy of afl-fuzz will take up one CPU core. This means that on an
n-core system, you can almost always run around n concurrent fuzzing jobs with
virtually no performance hit (you can use the afl-gotcpu tool to make sure).

每个 afl-fuzz 实例需要 1 个 CPU 核，这意味着在 n 核系统上，您几乎总是可以良好的并发运行 n 个
fuzzing 作业（使用 afl-gotcpu 工具了解 CPU 使用情况）

In fact, if you rely on just a single job on a multi-core system, you will
be underutilizing the hardware. So, parallelization is usually the right
way to go.

事实上，如果仅在多核系统上运行单一的 fuzzing 作业，您将无法充分利用硬件资源。因此，并行化通常是正确
的做法。

When targeting multiple unrelated binaries or using the tool in "dumb" (-n)
mode, it is perfectly fine to just start up several fully separate instances
of afl-fuzz. The picture gets more complicated when you want to have multiple
fuzzers hammering a common target: if a hard-to-hit but interesting test case
is synthesized by one fuzzer, the remaining instances will not be able to use
that input to guide their work.

当测试多个不相关的目标或在 dumb 模式下使用工具时，启动多个完全独立的 afl-fuzz 实例是没有任何问题
的。但如果启动多个模糊器测试同一个目标，情况就会变得复杂。即：如果其中一个模糊器生成了一个罕见但有趣
的测试用例，但其它模糊器并不能使用该测试用例。

To help with this problem, afl-fuzz offers a simple way to synchronize test
cases on the fly.

为了解决这个问题，afl-fuzz 提供了一种动态同步测试用例的简单方法。

2) Single-system parallelization
单系统并行测试
--------------------------------

If you wish to parallelize a single job across multiple cores on a local
system, simply create a new, empty output directory ("sync dir") that will be
shared by all the instances of afl-fuzz; and then come up with a naming scheme
for every instance - say, "fuzzer01", "fuzzer02", etc. 

要在本地系统并行化 fuzzing 作业，只需创建一个空的输出目录（同步目录），该目录会被所有 afl-fuzz 实
例共享。然后，为每个实例起个唯一的代号，如：fuzzer01、fuzzer02 等。

Run the first one ("master", -M) like this:

运行主实例（-M）：

$ ./afl-fuzz -i testcase_dir -o sync_dir -M fuzzer01 [...other stuff...]

...and then, start up secondary (-S) instances like this:

运行多个次实例（-S）：

$ ./afl-fuzz -i testcase_dir -o sync_dir -S fuzzer02 [...other stuff...]
$ ./afl-fuzz -i testcase_dir -o sync_dir -S fuzzer03 [...other stuff...]

Each fuzzer will keep its state in a separate subdirectory, like so:

每个模糊器在各自独立的子目录里维护其状态：

  /path/to/sync_dir/fuzzer01/

Each instance will also periodically rescan the top-level sync directory
for any test cases found by other fuzzers - and will incorporate them into
its own fuzzing when they are deemed interesting enough.

每个实例还将定期扫描同步目录，并将其它模糊器实例发现的有趣的测试用例同步到自己的测试用例集中。

The difference between the -M and -S modes is that the master instance will
still perform deterministic checks; while the secondary instances will
proceed straight to random tweaks. If you don't want to do deterministic
fuzzing at all, it's OK to run all instances with -S. With very slow or complex
targets, or when running heavily parallelized jobs, this is usually a good plan.

-M 和 -S 模式的区别在于主实例会执行确定性策略，而次实例则直接执行随机性策略。如果不想进行确定性 
fuzzing 作业，可以使用 -S 模式运行所有实例。对于非常慢和复杂的目标，或者运行高度并行化的作业时，使
用 -S 模式运行所有实例通常是一个不错的办法。

Note that running multiple -M instances is wasteful, although there is an
experimental support for parallelizing the deterministic checks. To leverage
that, you need to create -M instances like so:

请注意，尽管有并行执行确定性策略的实验支持，但运行多个主实例是浪费的。要使用的话，参照如下方式：

$ ./afl-fuzz -i testcase_dir -o sync_dir -M masterA:1/3 [...]
$ ./afl-fuzz -i testcase_dir -o sync_dir -M masterB:2/3 [...]
$ ./afl-fuzz -i testcase_dir -o sync_dir -M masterC:3/3 [...]

...where the first value after ':' is the sequential ID of a particular master
instance (starting at 1), and the second value is the total number of fuzzers to
distribute the deterministic fuzzing across. Note that if you boot up fewer
fuzzers than indicated by the second number passed to -M, you may end up with
poor coverage.

其中，冒号后的第一个值是特定主实例的顺序编号（从1开始），第二个值是主实例总数。注意，如果实际启动的主
实例少于这里指示的总数，最终的覆盖范围可能很差。

You can also monitor the progress of your jobs from the command line with the
provided afl-whatsup tool. When the instances are no longer finding new paths,
it's probably time to stop.

您还可以使用命令行工具 afl-whatsup 监控作业的进度，当不再有新的执行路径被发现时，可能就该停止了。

WARNING: Exercise caution when explicitly specifying the -f option. Each fuzzer
must use a separate temporary file; otherwise, things will go south. One safe
example may be:

警告：显式指定 -f 选项请务必小心，确保每个模糊器实例使用单独的临时文件。一个安全的例子如下：

$ ./afl-fuzz [...] -S fuzzer10 -f file10.txt ./fuzzed/binary @@
$ ./afl-fuzz [...] -S fuzzer11 -f file11.txt ./fuzzed/binary @@
$ ./afl-fuzz [...] -S fuzzer12 -f file12.txt ./fuzzed/binary @@

This is not a concern if you use @@ without -f and let afl-fuzz come up with the
file name.

如果使用 @@ 但不指定 -f 选项，让 afl-fuzz 自行处理，则不会有问题。

3) Multi-system parallelization
多系统并行测试
-------------------------------

The basic operating principle for multi-system parallelization is similar to
the mechanism explained in section 2. The key difference is that you need to
write a simple script that performs two actions:

多系统并行化的基本操作原理和单系统类似。关键区别在于需编写一个简单脚本处理如下两件事情：

  - Uses SSH with authorized_keys to connect to every machine and retrieve
    a tar archive of the /path/to/sync_dir/<fuzzer_id>/queue/ directories for
    every <fuzzer_id> local to the machine. It's best to use a naming scheme
    that includes host name in the fuzzer ID, so that you can do something
    like:

    使用 authorized_keys 认证方式的 SSH 连接到每台计算机，并将计算机本地的 afl-fuzz 实例生
    成的测试用例目录（/path/to/sync_dir/<fuzzer_id>/queue/）归档。

    for s in {1..10}; do
      ssh user@host${s} "tar -czf - sync/host${s}_fuzzid*/[qf]*" >host${s}.tgz
    done

  - Distributes and unpacks these files on all the remaining machines, e.g.:

    在每台计算机上分发解压其它计算机上的测试用例，如：

    for s in {1..10}; do
      for d in {1..10}; do
        test "$s" = "$d" && continue
        ssh user@host${d} 'tar -kxzf -' <host${s}.tgz
      done
    done

There is an example of such a script in experimental/distributed_fuzzing/;
you can also find a more featured, experimental tool developed by
Martijn Bogaard at:

在目录 experimental/distributed_fuzzing/ 下有一个此类脚本的示例。如下是一个功能更丰富的类似工
具：

  https://github.com/MartijnB/disfuzz-afl

Another client-server implementation from Richo Healey is:

另一个 C/S 实现的工具：

  https://github.com/richo/roving

Note that these third-party tools are unsafe to run on systems exposed to the
Internet or to untrusted users.

请注意，在互联网或不受信任的用户系统上运行这些第三方工具是不安全的。

When developing custom test case sync code, there are several optimizations
to keep in mind:

开发自定义测试用例同步工具时，需要记住一些优化：

  - The synchronization does not have to happen very often; running the
    task every 30 minutes or so may be perfectly fine.

    同步无需过于频繁，30分钟是个不错的时间间隔。

  - There is no need to synchronize crashes/ or hangs/; you only need to
    copy over queue/* (and ideally, also fuzzer_stats).

    无需同步 crashes/ 和 hangs/ 目录，只需同步 queue/ 目录下的测试用例（理想情况下，也可以同步    
    fuzzer_stats 文件）。

  - It is not necessary (and not advisable!) to overwrite existing files;
    the -k option in tar is a good way to avoid that.

    没有必要也不建议覆盖现有文件，tar 中的 -k 选项是避免这种情况的好方法。

  - There is no need to fetch directories for fuzzers that are not running
    locally on a particular machine, and were simply copied over onto that
    system during earlier runs.

    不需要为不在特定机器上本地运行的模糊器获取目录，而是在早期运行期间简单地复制到该系统上。

  - For large fleets, you will want to consolidate tarballs for each host,
    as this will let you use n SSH connections for sync, rather than n*(n-1).

    You may also want to implement staged synchronization. For example, you
    could have 10 groups of systems, with group 1 pushing test cases only
    to group 2; group 2 pushing them only to group 3; and so on, with group
    eventually 10 feeding back to group 1.

    This arrangement would allow test interesting cases to propagate across
    the fleet without having to copy every fuzzer queue to every single host.

    可以实现自定义的同步策略。

  - You do not want a "master" instance of afl-fuzz on every system; you should
    run them all with -S, and just designate a single process somewhere within
    the fleet to run with -M.

    只运行1个主实例，其它使用 -S 模式运行次实例。

It is *not* advisable to skip the synchronization script and run the fuzzers
directly on a network filesystem; unexpected latency and unkillable processes
in I/O wait state can mess things up.

不建议跳过同步脚本并直接在网络文件系统上运行模糊器，I/O 等待状态下的意外延迟和不可终止的进程可能会造
成混乱。

4) Remote monitoring and data collection
远程监控和数据收集
----------------------------------------

You can use screen, nohup, tmux, or something equivalent to run remote
instances of afl-fuzz. If you redirect the program's output to a file, it will
automatically switch from a fancy UI to more limited status reports. There is
also basic machine-readable information always written to the fuzzer_stats file
in the output directory. Locally, that information can be interpreted with
afl-whatsup.

可以使用 screen,nohup,tmux 等工具运行 afl-fuzz 实例，如果将程序输出重定向到文件，运行过程将从精
美的 UI 切换到有限的报告。模糊器基本的状态信息会写入输出目录下的 fuzzer_stats 文件中，
afl-whatsup 工具展示的数据便来自于该文件。

In principle, you can use the status screen of the master (-M) instance to
monitor the overall fuzzing progress and decide when to stop. In this
mode, the most important signal is just that no new paths are being found
for a longer while. If you do not have a master instance, just pick any
single secondary instance to watch and go by that.

原则上，您可以使用主（-M）实例的状态屏幕来监控总体模糊测试进度并决定何时停止。如果在较长一段时间内没
有发现新的执行路径，则可以中断测试。如果没有主实例，采用同样方法任意选择一个次实例操作即可。

You can also rely on that instance's output directory to collect the
synthesized corpus that covers all the noteworthy paths discovered anywhere
within the fleet. Secondary (-S) instances do not require any special
monitoring, other than just making sure that they are up.

您还可以通过实例的输出目录收集语料库，该语料库覆盖了所有值得注意的执行路径。次 (-S) 实例不需要任何特
殊监控，只需确保它们正常运行即可。

Keep in mind that crashing inputs are *not* automatically propagated to the
master instance, so you may still want to monitor for crashes fleet-wide
from within your synchronization or health checking scripts (see afl-whatsup).

请记住，崩溃输入不会自动同步到主实例，因此仍然需要从同步或运行状况检查脚本中监控整个队列的崩溃。

5) Asymmetric setups
不常见用法
--------------------

It is perhaps worth noting that all of the following is permitted:

  - Running afl-fuzz with conjunction with other guided tools that can extend
    coverage (e.g., via concolic execution). Third-party tools simply need to
    follow the protocol described above for pulling new test cases from
    out_dir/<fuzzer_id>/queue/* and writing their own finds to sequentially
    numbered id:nnnnnn files in out_dir/<ext_tool_id>/queue/*.

  - Running some of the synchronized fuzzers with different (but related)
    target binaries. For example, simultaneously stress-testing several
    different JPEG parsers (say, IJG jpeg and libjpeg-turbo) while sharing
    the discovered test cases can have synergistic effects and improve the
    overall coverage.

    (In this case, running one -M instance per each binary is a good plan.)

  - Having some of the fuzzers invoke the binary in different ways.
    For example, 'djpeg' supports several DCT modes, configurable with
    a command-line flag, while 'dwebp' supports incremental and one-shot
    decoding. In some scenarios, going after multiple distinct modes and then
    pooling test cases will improve coverage.

  - Much less convincingly, running the synchronized fuzzers with different
    starting test cases (e.g., progressive and standard JPEG) or dictionaries.
    The synchronization mechanism ensures that the test sets will get fairly
    homogeneous over time, but it introduces some initial variability.