2 years ago

#57679

test-img

Tambarskjelve

Memory increase when scrolling NSCollectionView images

I'm trying to display pdf pages in a collection view, but accumulate memory in the magnitude of several gigabytes when I scroll the view. The application itself is pretty basic, it's using PDFKit to preload every page of a pdf file as NSImage objects and stores them in an array indexed by an NSCollectionView. The view controller implements the NSCollectionViewDataSource protocol, and set the rendered pdf image from the array as the NSCollectionViewItem image whenever collectionView:itemForRepresentedObjectAtIndexPath: for that index is called. The problem seems to be rooted in the way the pages are rendered to images. There are no memory issues when I use thumbnailOfSize:forBox: to generate NSImages. The application does this concurrently in the background, but the same result can be achieved (for the sake of readability) when it loops on the main thread.

// _width, _height are determined by the pdf
CGSize pdfSize = CGSizeMake(_width, _height);
    
for (int i = 0; i < _pageCount; i++)
{        
    PDFPage *page = [_document pageAtIndex:i];
    NSImage *pdfImage = [page thumbnailOfSize:pdfSize forBox:kPDFDisplayBoxArtBox];
    [_pages replaceObjectAtIndex:i withObject:pdfImage];
}

With this method the application tops around 150 MB in memory usage, also when scrolling. This effectively solves the problem, but my issue with this method is that the the weight of the rendered typeface is a bit too light for the post-processing I intend to do, so I would like to use drawWithBox:toContext: instead. Sadly, that method triggers the memory issue.

// _width, _height are determined by the pdf
CGSize pdfSize = CGSizeMake(_width, _height);
CGRect pdfRect = CGRectMake(0.0f, 0.0f, _width, _height);

CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef gfxcontext = CGBitmapContextCreate(nil, (size_t)_width, (size_t)_height, 8, 0, colorSpace, kCGImageAlphaPremultipliedLast);

CGContextSetRGBFillColor(gfxcontext, 1.0f, 1.0f, 1.0f, 1.0f);
CGContextSetInterpolationQuality(gfxcontext, kCGInterpolationHigh);

for (int i = 0; i < _pageCount; i++)
{
    PDFPage *page = [_document pageAtIndex:i];
    CGContextFillRect(gfxcontext, pdfRect);
    [page drawWithBox:kPDFDisplayBoxBleedBox toContext:gfxcontext];
    CGImageRef cgPdfImage = CGBitmapContextCreateImage(gfxcontext);
    NSImage *nsPdfImage = [[NSImage alloc] initWithCGImage:newImage size:pdfSize];
    [_pages replaceObjectAtIndex:i withObject:nsimage];
    CGImageRelease(newImage);
}

CGColorSpaceRelease(colorSpace);
CGContextRelease(gfxcontext);

Now I can watch the application memory increase from 150 MB to several hundred MB in a few scrolls, and easily well over a GB if I continue scrolling. Profiling allocations in Instruments reveals the issue when comparing the two methods, and the major difference is that QuartzCore allocates substantial chunks of memory (assuming image data) when I scroll the page.

Allocations in instruments

It seems that mmap is what actually causes the memory increase. From what I understand, mmap is a Unix system call that implements "demand paging" which is obviously what I want, and likely works as intended in the first method. The image data seems to be retained in memory in the second method instead of being paged. The associated assembly may or may not be relevant.

+0x00   pushq               %rbp
+0x01   movq                %rsp, %rbp
+0x04   pushq               %r15
+0x06   pushq               %r14
+0x08   pushq               %r12
+0x0a   pushq               %rbx
+0x0b   testq               %rsi, %rsi
+0x0e   je                  "mmap+0x6e"
+0x10   movl                %ecx, %r12d
+0x13   movl                %ecx, %eax
+0x15   andl                $3, %eax
+0x18   je                  "mmap+0x6e"
+0x1a   movl                %r8d, %ebx
+0x1d   movq                %rsi, %r14
+0x20   movl                %r12d, %ecx
+0x23   orl                 $262144, %ecx
+0x29   callq               "0x7ff81b738594"
+0x2e   movq                %rax, %r15
+0x31   leaq                240919(%rip), %rax
+0x38   movq                (%rax), %rax
+0x3b   testq               %rax, %rax
+0x3e   je                  "mmap+0x7f"
+0x40   andl                $4278190080, %ebx
+0x46   orl                 $16, %ebx
+0x49   btl                 $12, %r12d
+0x4e   movl                $144, %edi
+0x53   cmovbl              %ebx, %edi
+0x56   leaq                256718(%rip), %rcx
+0x5d   movl                (%rcx), %esi
+0x5f   movq                %r14, %rdx
+0x62   xorl                %ecx, %ecx
+0x64   movq                %r15, %r8
+0x67   xorl                %r9d, %r9d
+0x6a   callq               *%rax
+0x6c   jmp                 "mmap+0x7f"
+0x6e   movl                $22, %edi
+0x73   callq               "0x7ff81b7382f1"
+0x78   movq                $-1, %r15
+0x7f   movq                %r15, %rax
+0x82   popq                %rbx
+0x83   popq                %r12
+0x85   popq                %r14
+0x87   popq                %r15
+0x89   popq                %rbp
+0x8a   retq

The Annotations view in Instruments simply says 100.00% +0x6c jmp "mmap+0x7f. My confusion is that the memory increase is caused by mmap during scrolling, but the problem seems to be rooted in the way NSImage is generated. I assume thumbnailOfSize:forBox: does a lot of optimization behind the curtains, while it is quite straightforward in the second method. What can I do to make the second method work?

Update

This looks like a bug. The memory issue is triggered by the colorspace used to generate image data. Setting the colorspace to NSScreen.mainScreen.colorSpace.CGColorSpace solves the memory issue in the second method, but can be reintroduced by connecting an external monitor while the application is running for both methods. Unlikely a feature.

objective-c

mmap

pdfkit

nsimage

nscollectionview

0 Answers

Your Answer

Accepted video resources