Exchange 2003: Virtual memory fragmentation errors (Event ID's 9582, 12800)

Written by - 0 comments

Published on - Listed in Windows Mail


Today I was informed by the 1st level support that they receive several calls from users who have problems to open their e-mails or attachments in Outlook. The following screenshot shows the error message shown in Outlook:

Outlook memory error

Translation: Impossible to open the element. Impossible to execute the operation, not enough memory.

I've tested this on my Outlook and the symptom was that I saw the list of e-mails but when I tried to open an e-mail message, the new window for the mail opened but the content of the mail was empty. However no error message was shown in my Outlook (2007).
Note: This only happened once I disabled the local cache in Outlook. Before that, with the local cache activated, mails were shown fine.

As this happened to many users at the same time, the source of the problem is easily located on the Exchange 2003 server.
Indeed, after a quick research in the Application Event Log, the following error was found:

Source: MSExchangeIS
Category: Performance
Event ID: 9582
Description: The virtual memory necessary to run your Exchange server is fragmented in such a way that normal operation may begin to fail. It is highly recommended that you restart all Exchange services to correct this issue.

According to KB 325044 this problem occors when the available space of the virtual memory of the Store.exe process (Information Store) declines. The thresholds are the following:

  • Less than 32MB: Warning in Application Log with Event ID 9582.
  • Less than 16MB: Error in Application Log with Event ID 9582. At this level, the Store.exe process cannot create additional heaps and cannot correctly mount and dismount Storage Groups.
  • Less than 10MB: Critical State where message operations may start to fail and message conversions problems occur. Storage Groups are not mounted anymore. This will affect users where they experience performance issues and they may not be able to access their messages. When such message processing issues occur, another error is logged in the Application Event Log (Event ID 12800).

Such a second Event Log entry was found just 4 minutes after the first error:

Source: MSExchangeIS
Category: Content Engine
Event ID: 12800
Description: Message processing failed because there is not enough available memory (8007000E-820003A2).

To temporarily resolve the problem, the MSExchangeIS service can be restarted. But this is only a temporary solution, as the virtual memory fragmentation will most likely return.

The way to handle this error more properly, the following steps should be made (maybe even more, do whatever is in your power to increase stability!):

  • Install all latest Service Packs and updates
  • Set the /3GB switch in the boot.ini - more information can be found in KB 823440
  • Set /userva switch in the boot.ini (with a value between 2970 and 3030, the latter is recommended)
  • Reboot the Exchange server at the next convenient moment (it should be done preferably on the same day)
  • Monitor the Exchange Server Application Event Log for such errors with Event ID's 9582 and 12800
  • Monitor the value of the performance counter VM Largest Block Size (in object MSExchangeIS) - this counter displays the size in bytes of the largest free block of virtual memory. It is this counter to watch and which will trigger the errors in the Event Log.
  • Monitor the value of the performance counter VM Total Large Free Block Bytes (in object MSExchangeIS) to determine the rate of decay of the virtual memory



Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.

RSS feed

Blog Tags:

  AWS   Android   Ansible   Apache   Apple   Atlassian   BSD   Backup   Bash   Bluecoat   CMS   Chef   Cloud   Coding   Consul   Containers   CouchDB   DB   DNS   Database   Databases   Docker   ELK   Elasticsearch   Filebeat   FreeBSD   Galera   Git   GlusterFS   Grafana   Graphics   HAProxy   HTML   Hacks   Hardware   Icinga   Influx   Internet   Java   KVM   Kibana   Kodi   Kubernetes   LVM   LXC   Linux   Logstash   Mac   Macintosh   Mail   MariaDB   Minio   MongoDB   Monitoring   Multimedia   MySQL   NFS   Nagios   Network   Nginx   OSSEC   OTRS   Office   PGSQL   PHP   Perl   Personal   PostgreSQL   Postgres   PowerDNS   Proxmox   Proxy   Python   Rancher   Rant   Redis   Roundcube   SSL   Samba   Seafile   Security   Shell   SmartOS   Solaris   Surveillance   Systemd   TLS   Tomcat   Ubuntu   Unix   VMWare   VMware   Varnish   Virtualization   Windows   Wireless   Wordpress   Wyse   ZFS   Zoneminder