System Crashing when Content Indexing is enabled

[Trevor22]

1.5 alpha with content indexing enabled causes my system to freeze. It uses all of the memory on my PC. Even after setting the "Maximum size" Content setting to something like 2 GB, the program still uses up over 10 GB of memory and then crashes my system. I've replicated this problem on several PCs. Also, I am using the "include only" file setting to minimize the number of files indexed.

Has anyone found a way to stop this from happening?

[void]

Everything stores content in memory.

Please try excluding more files with content filters under Tools -> Options -> Content.

The Maximum size is per file (not the content index size)

Content indexing in Everything is designed for indexing a few user documents (< 1 GB of text)



10 GB is a lot of text.
What file extensions are you indexing?
Everything uses system iFilters to read text from non-text files.
It's possible a shell extension is getting stuck reading content here.
Please try limiting your content indexing to text/plain files only and see if the issue persists.

[Trevor22]

Please try limiting your content indexing to text/plain files only and see if the issue persists.

I just tried your suggestion. When I limit to text files, it works fine, but when I add in .pdf and .docx, it consumes all available memory on the machine and then locks up.

[void]

Please try including docx files.

Does the issue persists?



What PDF viewer do you have?



Everything will show the current pdf/docx file being index under Tools -> Options -> Content.
Is Everything getting stuck on a particular file?



Please make sure you are using the x64 version of Everything

[Trevor22]

It still locks up after adding docx. As files are being indexed, the memory usage seems to gradually climb and then eventually the PC runs out of memory, but it doesn't seem to get stuck on any file in particular.

I'm using Adobe Acrobat Pro XI .

Thanks again

[tuska]

Everything stores content in memory.
...
Content indexing in Everything is designed for indexing a few user documents (< 1 GB of text)
10 GB is a lot of text.

2Trevor22
Below is an example of how I handle this issue and to show how quickly the 1 GB limit RAM is reached/exceeded.
 

2022-12-18_Everything Content indexing and Task Manager - Processes.png (59.94 KiB) Viewed 12351 times

 
Content indexed: ~ 35,000 files ... Size: 16 GB

Code: Select all

*.txt;*.pdf;*.doc;*.docx;*.xls;*.xlsm;*.xlsx;*.csv;*.md;*.ion;*.ini;*.eml;*.odt
ext:txt;pdf;doc;docx;xls;xlsm;xlsx;csv;md;ion;ini;eml;odt

Hereof

Code: Select all

ext:pdf  ~ 3,500 files ... Size: 14 GB
ext:docx ~ 2,500 files ... Size: 65 MB

In addition to the documents mentioned above, I also index a few file properties.

For all other file types that are to be searched for content, I use Windows Search.
In Everything it is possible to access Windows Search with the parameter si: and display the search result in Everything.
It is highly recommended to set the Windows Search indexing options accurately.
 

2022-12-18_Windows Search settings - Advanced indexing options.png (57.26 KiB) Viewed 12351 times

 
________________________________________________________________________________________________
Windows 11 Pro (x64) Version 22H2 (OS build 22621.963) | Adobe Acrobat 11 Pro - Version 11.0.23
'Everything' 1.5.0.1331a (x64) | C:\Everything\Everything64.exe - "Portable on system drive C:"
Microsoft® Outlook® LTSC MSO (16.0.14332.20431) 64-Bit (Part of: Microsoft Office LTSC Professional Plus 2021)
Processor Intel(R) Core(TM) i5-12600K 12th Gen, 3.70 GHz, 10 Cores, 16 Logical Processors
Installed RAM 32.0 GB (31.8 GB usable) | System type 64-bit operating system, x64-based processor

[Trevor22]

2Trevor22
Below is an example of how I handle this issue and to show how quickly the 1 GB limit RAM is reached/exceeded.
 

Thanks for this. Since Everything index seems to use more memory than Windows Search Index, why do you use both indexes instead of only using Windows Search Index?

[void]

Thank you for the information Trevor22,

I thought there might be a memory leak here for docx files.
However, I haven't been able to produce a memory leak my end.

How many docx files are you content indexing? (search for: *.docx )
What is the size of all your docx files? (search for: *.docx -total size is shown in the status bar on the bottom right)

[void]

If the system iFilter fails to load the docx file, then Everything will treat the content as text/plain.
This will most likely lead to undesired results and over-allocation for each docx file.
-The next alpha update will no longer treat file types with registered iFilters as text/plain.



Are you running Everything as an administrator?
Is [Administrator] shown in the window title bar?

Please make sure Everything is installed correctly and running as a standard user:

• In Everything, from the Tools menu, click Options.
• Click the General tab on the left.
• Check Store settings and data in %APPDATA%\Everything.
• Uncheck Run as administrator.
• Check Everything Service. (Please make sure this is tick-checked and not square-checked)
• Click OK.
• Exit Everything (right click the Everything tray icon and click Exit).
• Restart Everything.

[Trevor22]

Thank you for the information Trevor22,

I thought there might be a memory leak here for docx files.
However, I haven't been able to produce a memory leak my end.

How many docx files are you content indexing? (search for: *.docx )
What is the size of all your docx files? (search for: *.docx -total size is shown in the status bar on the bottom right)

16,800 files, .97 GB total size.

[Trevor22]

Are you running Everything as an administrator?
Is [Administrator] shown in the window title bar?

It's not being run as administrator.

I checked the settings as you instructed, and they were already set to the selections you recommended.

Thanks again

[tuska]

2Trevor22
Below is an example of how I handle this issue and to show how quickly the 1 GB limit RAM is reached/exceeded.

Since Everything index seems to use more memory than Windows Search Index,
why do you use both indexes instead of only using Windows Search Index?

Those are my reasons, viz
- as a rule, Everything shows me the search result a little faster with my/these settings
- sometimes I had different search results (Everything | Windows Search) -
  as a rule, however, the results match!
- Thus, a second search query also helps me to check an unexpected or no search result.
- (But I also set exceptions to avoid double indexing).

[void]

Everything 1.5.0.1332a will no longer treat files as text/plain when the iFilter fails to read content.

Please try this version and let us know if the issue persists.

16,800 files, .97 GB total size.

Everything should be using well under 1GB of RAM for content here.

[Trevor22]

Everything 1.5.0.1332a will no longer treat files as text/plain when the iFilter fails to read content.

Please try this version and let us know if the issue persists.

I tried this version, and it still crashes.

16,800 files, .97 GB total size.

Everything should be using well under 1GB of RAM for content here.

It was just slightly under 1 GB (.97 GB)

[void]

Thank you reporting the crash.

We should now look at a mini crash dump / debug logs to work out what file is causing the crash.

I suspect the system iFilter for docx files is causing Everything to crash.

Could you please enable verbose debug logging and re-enable content indexing:

• In Everything, from the Tools menu, under the Debug submenu, check Verbose.
• From the Tools menu, under the Debug submenu, check Start Debug Logging.
• From the Tools menu, click Options.
• Click the Content tab on the left.
• Re-enable content indexing.
• Click OK.

When Everything crashes, could you please send a mini crash dump:

• Keep the crash dialog open.
• Download Process Explorer from Microsoft:
  https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer
• In Process Explorer, select Everything64.exe (If there is two, select the one using the most memory)
• Right click Everything64.exe and under Create Dump, select Create Mini Dump.
• Choose a filename and click Save.
• Please send this file to support@voidtools.com

Your %TEMP%\Everything Debug Log.txt will also contain information about the file currently being content indexed.
Could you please send this file to support@voidtools.com

Thank you for your help with troubleshooting the issue.


Privacy

[Trevor22]

ould you please enable verbose debug logging and re-enable content indexing:

i've created the two crash/debut files. I assume these files contain a list of on my PC, which contains some confidential info. Do you have a more secure method that I can use to send this to you and/or know of a way to obscure the confidential information?


Thanks

[void]

The mini crash dumps should only be a couple MB.
The mini crash dump will contain some memory used by Everything (not a complete snap shot)

There might be a few filenames on the stack stored in the mini crash dump.

The debug logs may contain filenames of recently changes files.

Please consider encrypting the files and pm me the key.

[Trevor22]

The crash dump does seem relatively small, but the log file is over 10 mb--too large for email. Is it possible that the crash dump will be sufficient without the log file? If not, do you have have a method for me to send a very large file to you?

Thanks

[void]

Is it possible that the crash dump will be sufficient without the log file?

Yes, the crash dump on its own will be helpful with figuring out the issue.

Don't worry about the debug log if it doesn't compress to under 10MB.

[Trevor22]

Thanks. Sent by email.

[void]

Thank you for the mini crash dump Trevor22,

The crash dump is from Everything 1.4.0.1017 and Everything appears to be running normally.



If you see Everything 1.5 crash again, could you please:

• Keep the crash dialog open.
• Download Process Explorer from Microsoft:
  https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer
• In Process Explorer, select Everything64.exe (If there is two, select the one using the most memory)
• Right click Everything64.exe and under Create Dump, select Create Mini Dump.
• Choose a filename and click Save.
• Please send this file to support@voidtools.com

(Everything.exe => Everything64.exe)

[Trevor22]

I waited for Everything 1.5 to crash, but when trying to create the crash dump for Everything64.exe, I received the message "error opening process: The handle is invalid" and it wouldn't allow me to create it.

[void]

Thanks for trying Trevor22,

The stack is most likely too corrupt, so Everything just exits.

Debug logs may help find the problem:

• In Everything, from the Tools menu, under the Debug submenu, check Verbose.
• From the Tools menu, under the Debug submenu, check Start Debug Logging.
• From the Tools menu, click Options.
• Click the Content tab on the left.
• Re-enable content indexing.
• Click OK.

Wait for Everything to crash.
What are the last few lines in your %TEMP%\Everything Debug Log.txt
Could you please send them to support@voidtools.com