On Tue, 27 Sep 2011, Chris Samuel wrote:
On 27/09/11 15:48, Tim Connors wrote:
It's just getting worse and worse as I upgrade kernels. The other machines are at 2.6.32, which was bad enough, but 3.0 seems absolutely terrible at virtual memory management.
I've seen the same, dropped back to 2.6.38 in Ubuntu after trying 3.0.x for a while to regain some sanity. Sigh..
Given how contiguous reads/writes off commodity hardware today yields data at about 80MB/s, and data being pulled in and out of swap is more like about 2MB/s because of seeks, I wonder why no one has thought to implement a cache flushing policy heuristic that preferentially evicts large contiguous blocks from the cache 40 times more readily than small blocks. Or evicts large blocks 40 times more readily than causing something to swap. Keep the current policy for blocks less than say, about 1 MB. Tunable of course (although, it would be far better if it self-tuned dynamically based on measured IO reponse times and data accessing patterns). No need to keep my HD movie in RAM from when my recorder recorded it 3 hours ago, given that when I come to read it in again in 3 weeks time, the few MB/s needed to read the compressed stream at real-time rates will be orders of magnitude less than my disk could provide. If it so desired, it could buffer an entire movie into memory in the order of a few tens of secs, but if it decided to dump the equivalent amount of data from mozilla out to swap, would take about half an hour. Funnily enough, the same heuristic will be just as useful for SSDs, given their crappy write performance for small writes. -- Tim Connors