python: PyPerf: Aarch64 support

[Jongy]

Description

Support PyPerf running on Aarch64.

This currently passes "sanity" - works on Python 2.7.18 on Ubuntu 20.04 kernel 5.13.0-1008-aws. I am opening this PR to document what's left to be done & the tests we should run on it.

This depends on:

• BCC PR: Aarch64 support Granulate/bcc#34
• Aarch64 support for https://github.com/Jongy/bpf_get_stack_offset/tree/aarch64
• Aarch64 support for https://github.com/Jongy/bpf_get_fs_offset/tree/aarch64

TODOs

• Fix Aarch64 Python3 support (only Py2 works)
• Fix Aarch64 native stacks
• Figure out BCC build - with LLVM shared or not - so it works both in container & exe Aarch64 and doesn't break x86_64.
• Create final tag in https://github.com/Jongy/bpf_get_stack_offset
  • make sure we scan at least 16kb of stack, as suggested here.
• Create final tag in https://github.com/Jongy/bpf_get_fs_offset
  • make sure we scan at least 16kb of stack, as suggested here.
• Figure out if we really depend on https://lore.kernel.org/bpf/796ee46e948bc808d54891a1108435f8652c6ca4.1572649915.git.daniel@iogearbox.net/#r for reading usermode memory. If so, do not attempt to run PyPerf on lower kernel versions.
• Clean up BCC PR, run tests alongside, then merge it :)

Testing

x86_64:

• x86_64 native stacks (as I have made changes in that area) when in user mode
• x86_64 native stacks (as I have made changes in that area) when in kernel mode
• x86_64 gProfiler as container PyPerf
• x86_64 gProfiler as exe PyPerf
• get_stack_offset on 4.14
• get_stack_offset on 4.18
• get_stack_offset on 5.11
• get_stack_offset on 5.16
• get_fs_offset on 4.14
• get_fs_offset on 4.18
• get_fs_offset on 5.11
• get_fs_offset on 5.16

Aarch64:

• Get gProfiler test suite to run on Aarch64 (specifically the Python tests pertaining PyPerf)
• Aarch64 gProfiler as container PyPerf
• Aarch64 gProfiler as exe PyPerf
• Aarch64 native stacks when in user mode
• Aarch64 native stacks when in kernel mode
• Aarch64 kernel stacks
• Aarch64 Python 2.7 Python stacks incl classes (glibc)
• Aarch64 Python 3.6 Python stacks incl classes (glibc)
• Aarch64 Python 3.7 Python stacks incl classes (glibc)
• Aarch64 Python 3.8 Python stacks incl classes (glibc)
• Aarch64 Python 3.9 Python stacks incl classes (glibc)
• Aarch64 Python 3.10 Python stacks incl classes (glibc)
• Aarch64 Python 3.8 Python stacks (musl based, one Python version is enough)

[Jongy]

x86_64: build should pass, and hopefully will the tests.

Aarch64: I managed to build container, exe still failing. Tested against Python 3.8 on kernel5.11, Python symbols all get [Error (errnos) 14] stacks and native stacks get [Error unw_init_remote (bad register number)]_[pn]. Kernel stacks work but these are easiest 🤷

[Jongy]

Got it running just fine against Python 2 (well, as the description says). Native stacks still fail, but I get good Python 2 symbols & including class names. Very good

[Jongy]

Python 3 works now

[Jongy]

Native stacks are broken on x86_64, I'll fix that and then continue to fixing it on Aarch64. So far on Aarch64 I'm seeing a similar error to libunwind/libunwind#113 but it's not clear if that error is transient or not.