Sorry, lots of context before the actual question as we've throughly researched this and I wanted to give you full context.
Some context: postgres index-only-scans rely on the visibility map (VM). If a page is not marked as not-fully-visible in the visibility map, postgres fetches that page to ensure the data is visible to this transaction, even when doing index only scans. Unfortunately, this can greatly slowdown index only scans. The index might return results from 10k rows, but the index itself only spans 50 pages (very fast in terms of IO). However, if VM isn't set, it makes and extra 10k heap fetches (200x slower in terms of IO).
Try it yourself: EXPLAIN ANALYZE an index only query, before and after a VACUUM. You can see the number of heap fetches go down after the VACUUM (assuming you had some dirty pages in the VM before)
Already tried: We're already tuned autovacuum, and we're vacuuming regularly. This helps a lot, but we'd like to get even faster.
Question (finally): Is it possible to skip heap fetches when doing index only scans? I'm aware we wouldn't have perfect MVCC when reading, but we're okay with that. The data in the index is close enough, and it's definitely not worth the overhead of thousands of heap fetches to make sure we're not looking at slightly stale data. To borrow a term from NoSQL, we'd be fine with "eventual consistency" reads.