Last modified by colinw on 2021/09/10 22:40
Show last authors
1 {{info}}
2 The information in this document applies to **iSymphony 3**.
3 {{/info}}
4
5 == Problem ==
6
7 iSymphony crashes, or becomes unstable, and a message similar to the following appears in /opt/isymphony3/server/log/iSymphonyServer.err.log:
8
9 {{code}}
10 Exception in thread "<threadname>" java.lang.OutOfMemoryError: Java heap space
11 {{/code}}
12
13 == Temporary Workaround ==
14
15 As a temporary workaround, restarting the iSymphony server will typically allow you to continue using the panel. However, it is unlikely to solve any underlying issues that caused the error in the first place, so you will likely run into the same problem again at some point in the future. See [[doc:ISYMDOCS.Starting And Stopping The iSymphony Server]] for details on restarting iSymphony, and see below for information about troubleshooting the actual problem to ensure it doesn't happen again.
16
17 == Troubleshooting ==
18
19 This error indicates that iSymphony has run out of memory and was unable to allocate more. There are several potential causes for this issue:
20
21 * The operating system on the server that iSymphony is installed on has run out of available memory, and was unable to allocate enough to iSymphony. In this case, you'll need to either free up memory on your server by stopping other processes that are running, or install/allocate more memory in the system itself.
22 * iSymphony has reached the limit of what it is allowed to use based on the current memory settings for the JVM, in a legitimate manner. This is unlikely unless you are running a very large system with a lot of call traffic.
23 * iSymphony is leaking memory and will gradually use more and more memory as time passes, until it hits the limit of what it is allowed to use.
24
25 See the following sections for details on determining which potential cause is the actual issue, and proceeding to troubleshoot further
26
27 === The operating system has run out of memory ===
28
29 To determine whether the operating system has run out of memory entirely, wait until iSymphony has stopped responding or is otherwise in an undesirable state. The run the following command on the server, and note the results:
30
31 {{code}}
32 $ free -m
33 total used free shared buffers cached
34 Mem: 5980 4856 1123 0 401 3381
35 -/+ buffers/cache: 1073 4906
36 Swap: 2863 31 2831
37 {{/code}}
38
39 In this example, the 5980 under the total column indicates that the system has approximately 6GB of memory installed. Of that, approximately 4.8GB is used (the 4856 under the used column), and approximately 1.1GB of physical RAM is available for use (the 1123 under the free column). It is important to note that after all physical ram has been used, the Linux operating system will then begin using swap space. If the values in the free column for both Mem and Swap are both very low (less than 100) or zero, then it is likely that the operating system is running out of memory, and will need more installed or allocated to continue working as it should. Alternatively, you can allocate more swap space, but this will likely have performance impacts on both iSymphony and any additional processes running on this system.
40
41 === iSymphony has reached the limit of what it is allowed to use ===
42
43 All Java applications, including iSymphony, have a limit on how much memory they are allowed to use, to prevent them from consuming excessive resources and affecting the other processes on the system. If iSymphony has reached its limit, it will trigger the same errors. On very heavy use systems, it is possible for iSymphony to reach this limit legitimately, in which case the problem can be solved simply by allowing iSymphony to use more memory. However, on lower traffic systems, it is significantly more likely that iSymphony is leaking memory in some manner, and increasing the memory it is allowed to use will only delay the problem. To further troubleshoot a memory leak, you will need to send a heap dump to the iSymphony team for further investigation. Here are instructions to acquire a heap dump:
44
45
46
47 {{panel bgColor="#fff" title="How to Acquire an iSymphony Heap Dump"}}
48 1. (((
49 Modify the file /opt/isymphony3/server/jvm/args, and add the following two lines to the end:
50
51 {{code}}
52 -XX:+HeapDumpOnOutOfMemoryError
53 -XX:HeapDumpPath=/opt/isymphony3/server/
54 {{/code}}
55 )))
56 1. Save and close this file, and restart iSymphony. See [[doc:ISYMDOCS.Starting And Stopping The iSymphony Server]] for instructions on restarting iSymphony.
57 1. Wait until iSymphony stops responding or otherwise crashes. At this point, if the crash is due to an OutOfMemoryError, a file will have been created in /opt/isymphony3/server, with the filename <processid>.hprof
58 1. Attach this file to a ticket with iSymphony support, and indicate that you experienced an OutOfMemoryError. If you have not already done so, please include your log files as well, as detailed in [[doc:ISYMDOCS.Reporting Issues]].
59 {{/panel}}
60
61 == Related articles ==
62
63
64
65 {{contentbylabel spaces="" showLabels="false" reverse="true" sort="modified" max="5" labels="outofmemory memory crash" showSpace="false" type="page"/}}

Search

Navigation

iSymphony