Skip to content

Tracking Issue for porting run-make tests to use Rust #121876

Closed

Description

@jieyouxu

We want to stop run-make tests from relying on make, and improve the run-make tests so that
they are more accessible to rustc contributors by allowing the tests to be written in Rust (see
#40713 for context). PR #113026 was merged to address this, and now we are able to write run-make
tests in Rust recipes. We would like your help to port over existing run-make tests still using
Makefiles to use Rust recipes instead.

If you would like to work on porting one of the run-make tests, please link to this issue in your
PR and leave a comment to claim the test (or multiple tests). If you are stuck, please don't hesitate to open a thread
on Rust's Zulip.

run-make is the most flexible fallback test kind, but some run-make tests could be migrated to become e.g. ui tests instead. Please check if you can convert the run-make test into other test kinds before porting!

When you try to port a test, also consider:

  • Add some comments on what the test is trying to test.
  • Add some comments on how the test tries to accomplish its goals if the test is non-trivial. This is especially true if you discovered that a test has platform/compiler/architecture/tooling-specific behavior that is a pain to debug.
  • Any related issues? Any relevant links?
  • Is the test still applicable? Has it become outdated? Is it duplicated?
  • Can it be written in other test suites?

Context: Rust recipes?

PR #113026 adds basic infrastructure support to write run-make tests using small Rust programs,
called recipes.

We aim to eliminate the dependency on make and Makefiles for building run-make-style tests.
Makefiles are replaced by recipes (rmake.rs). The PR implements running run-make recipes in
3 steps:

  1. We build the support library run_make_support which the rmake.rs recipes depend on as a tool
    lib.
  2. We build the recipe rmake.rs and link in the support library.
  3. We run the recipe to build and run the tests.

rmake.rs is basically a replacement for Makefile, and allows running arbitrary Rust code. The
support library is built using cargo, and so can depend on external crates if desired.

The infrastructure implemented by the PR is very barebones, and is the minimally required
infrastructure needed to build, run and pass the two example run-make tests ported over to the new
infrastructure:

You likely will find that you would need to improve the API of the support library, and extend
the functionality of the support library.

Common traps and pitfalls, and tips and tricks

  • $PATH uses ; on Windows and : for *nixes. Use std::env::{join,split}_paths to properly
    handle $PATH.

  • Be careful of path separator platform differences. Always prefer PathBuf operations not string
    paths if possible.

  • tests/ are not (currently) formatted by rustfmt.

  • Consult https://github.com/rust-lang/rust/blob/master/tests/run-make/tools.mk for which flags
    and envs are passed to various executables or libraries. May have to triple check on those.

  • You can always request your reviewer to run try jobs to test out your PR on environments you don't locally have access to. Good candidate CI jobs include:

    • aarch64-apple (apple, 64 bits, aarch64/arm64)
    • armhf-gnu (cross-compile)
    • test-various (cross-compile, wasm)
    • x86_64-mingw (x86_64, 64 bits, windows, mingw)
    • x86_64-msvc (x86_64, 64 bits, windows, msvc)
    • x86_64-gnu-llvm-18 (x86_64, 64 bits, windows, gnu, llvm 18)
    • i686-msvc (x86, 32 bits, windows, msvc)

Tests that need porting

Activity

  1. added
    A-testsuiteArea: The testsuite used to check the correctness of rustc
    T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
    E-help-wantedCall for participation: Help is requested to fix this issue.
    C-tracking-issueCategory: An issue tracking the progress of sth. like the implementation of an RFC
    on Mar 1, 2024
  2. abhay-51 commented on Mar 2, 2024

    @abhay-51
  3. high-cloud commented on Mar 13, 2024

    @high-cloud
  4. 0xhiro commented on Mar 13, 2024

    @0xhiro
  5. JoverZhang commented on Mar 18, 2024

    @JoverZhang
  6. added
    E-mentorCall for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion.
    on Mar 22, 2024
  7. Oneirical commented on Mar 29, 2024

    @Oneirical
  8. Oneirical commented on Mar 31, 2024

    @Oneirical
  9. added a commit that references this issue on Mar 31, 2024
  10. added a commit that references this issue on Mar 31, 2024
  11. added a commit that references this issue on Apr 4, 2024
  12. 317 remaining items

  13. lolbinarycat commented on Jul 28, 2024

    @lolbinarycat
    Contributor

    claiming symbol-mangling-hashed

  14. Oneirical commented on Jul 29, 2024

    @Oneirical
    Contributor

    incr-prev-body-beyond-eof and suspicious-library are already ported, they should be checked off.

  15. Kobzol commented on Jul 29, 2024

    @Kobzol
    Member

    Thanks, fixed. They were included twice in the list.

  16. Oneirical commented on Jul 30, 2024

    @Oneirical
    Contributor

    rust-lld and rust-lld-custom-target are already ported, they should be checked off.

  17. Kobzol commented on Jul 30, 2024

    @Kobzol
    Member

    Thanks, fixed.

  18. lolbinarycat commented on Aug 2, 2024

    @lolbinarycat
    Contributor

    cat-and-grep-sanity-check can probably be deleted along with tools.mk once migration is complete

  19. Oneirical commented on Aug 12, 2024

    @Oneirical
    Contributor

    valid-print-requests in the checklist does not exist.

  20. Kobzol commented on Aug 12, 2024

    @Kobzol
    Member

    Thanks! It was turned into a UI test in #124333.

  21. Zalathar commented on Aug 13, 2024

    @Zalathar
    Member

    Explicitly claiming libtest-junit alongside libtest-json, since they're very similar, and I've decided to go with a straightforward rmake port for now.

  22. petrochenkov commented on Aug 21, 2024

    @petrochenkov
  23. jieyouxu commented on Aug 22, 2024

    @jieyouxu
    Author
  24. Zalathar commented on Oct 12, 2024

    @Zalathar
  25. jieyouxu commented on Nov 7, 2024

    @jieyouxu
    Author
  26. jieyouxu commented on Dec 25, 2024

    @jieyouxu
    Author
  27. jieyouxu commented on Jan 16, 2025

    @jieyouxu
    MemberAuthor

    Re-triage (2025-01-23):

    Test Latest PR Status
    split-debuginfo #135572 New PR #135572

    Also need to:

    • Remove tools.mk after all Makefiles are gone.
    • Remove support for legacy Makefile run-make infra
    • Update run-make docs to remove any mention of the Makefile infra
    • Remove tidy check for Makefiles once all of them are gone.

    Implemented in: #136581

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

A-run-makeArea: port run-make Makefiles to rmake.rsA-testsuiteArea: The testsuite used to check the correctness of rustcC-tracking-issueCategory: An issue tracking the progress of sth. like the implementation of an RFCT-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.

Type

No type

Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions