replay on device hangs

[sshane]

journalctl -k -n 200 --no-pager

Dec 20 04:14:56 comma-8c64008d kernel: kworker/u16:2   D    0   324      2 0x00000000
Dec 20 04:14:56 comma-8c64008d kernel: Workqueue: pm_workerq_venus venus_hfi_pm_handler
Dec 20 04:14:56 comma-8c64008d kernel: Call trace:
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df5286bb0>] __switch_to+0xb0/0xe0
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df66d8520>] __schedule+0x200/0x728
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df66d8d00>] schedule+0x38/0x98
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df66d9324>] schedule_preempt_disabled+0x1c/0x30
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df66dabcc>] __mutex_lock_slowpath+0xac/0x1a8
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df66dad14>] mutex_lock+0x4c/0x50
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df5d22f98>] venus_hfi_pm_handler+0xe0/0x970
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df52cb7cc>] process_one_work+0x20c/0x4d8
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df52cbae8>] worker_thread+0x50/0x4d0
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df52d2a90>] kthread+0x100/0x108
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df5283f00>] ret_from_fork+0x10/0x50
Dec 20 04:14:56 comma-8c64008d kernel: kworker/u16:8   D    0  4450      2 0x00000000
Dec 20 04:14:56 comma-8c64008d kernel: Workqueue: kgsl_devfreq_wq do_devfreq_notify
Dec 20 04:14:56 comma-8c64008d kernel: Call trace:
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df5286bb0>] __switch_to+0xb0/0xe0
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df66d8520>] __schedule+0x200/0x728
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df66d8d00>] schedule+0x38/0x98
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df66db4f8>] __rt_mutex_slowlock+0x58/0x100
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df66db658>] rt_mutex_slowlock+0xb8/0x1b0
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df66db7b4>] rt_mutex_lock+0x64/0x68
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df57b0834>] update_request_adhoc+0x34/0x440
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df57a9efc>] msm_bus_scale_client_update_request+0x34/0x68
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df62f753c>] devbw_target+0x11c/0x1a0
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df62e93fc>] update_devfreq+0xec/0x1f8
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df62eda6c>] devfreq_vbif_update_bw+0x5c/0xb0
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df599ce20>] kgsl_pwrctrl_buslevel_update+0xb8/0x248
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df59a4150>] kgsl_busmon_target+0xf0/0x178
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df62e93fc>] update_devfreq+0xec/0x1f8
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df62eccb4>] tz_notify+0x84/0xa8
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df52d4594>] notifier_call_chain+0x5c/0x98
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df52d4aa0>] __srcu_notifier_call_chain+0x78/0xb8
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df52d4b1c>] srcu_notifier_call_chain+0x3c/0x50
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df59a305c>] do_devfreq_notify+0x2c/0x38
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df52cb7cc>] process_one_work+0x20c/0x4d8
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df52cbae8>] worker_thread+0x50/0x4d0
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df52d2a90>] kthread+0x100/0x108
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df5283f00>] ret_from_fork+0x10/0x50
Dec 20 04:14:56 comma-8c64008d kernel: python3 ./selfd D    0 49776  49143 0x00000001
Dec 20 04:14:56 comma-8c64008d kernel: Call trace:
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df5286bb0>] __switch_to+0xb0/0xe0
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df66d8520>] __schedule+0x200/0x728
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df66d8d00>] schedule+0x38/0x98
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df66d9324>] schedule_preempt_disabled+0x1c/0x30
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df66dabcc>] __mutex_lock_slowpath+0xac/0x1a8
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df66dad14>] mutex_lock+0x4c/0x50
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df59b9ea8>] sendcmd+0x60/0x5b0
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df59bbf50>] _adreno_dispatcher_issuecmds.part.9+0x180/0x6e8
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df59bd8e4>] adreno_dispatcher_queue_cmds+0x884/0x918
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df598caa8>] kgsl_ioctl_gpu_command+0x1c8/0x368
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df599a214>] kgsl_ioctl_helper+0xac/0x170
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df599a328>] kgsl_ioctl+0x50/0xf0
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df545ee9c>] do_vfs_ioctl+0xc4/0x810
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df545f678>] SyS_ioctl+0x90/0xa8
Dec 20 04:14:56 comma-8c64008d kernel: [<ffffff8df5283f80>] el0_svc_naked+0x34/0x38
Dec 20 04:14:56 comma-8c64008d kernel: replay          D    0 50165  49776 0x00000000

[sshane]

I was able to reproduce this by launching replay with Popen inside __init__ function of MiciMainLayout. In the render function, it doesn't seem to happen

[greatgitsby]

is this because of the selfdriveState publisher in mici codepath?

maybe try on 3x and four and see if 3x hangs.