No. Audible is not only for heavy readers, but it is easiest to justify when audiobooks are already a regular habit. Occasional listeners can still use it, but the value case gets thinner if you go long stretches without finishing books or using the included membership benefits.
The reason this misconception survives is that Audible does have a clear centre of gravity. Its strongest case is still people who actually finish audiobooks often enough for a membership structure to make sense, which is why people reduce it to a narrower stereotype than it really deserves.