Filemaker Server 13 randomly crashes, mostly while doing scheduled backups (25 Comments)

Issue report by pajonate

Product

FileMaker Server

Version

Current Installed Version: 13.0.5.520

Operating system version

10.8.5

Description of the issue

Hi,
let me join the group of people that are reporting an issue with FMS crashing randomly.
My company is one of the bigger Filemaker customers. We have approximately 200 users using the system all the time, 6 days a week. Our business is a retail store, so we write to the system consistently.
We have a connection with mySQL/PHP part of the system, where we replicate a lot of the FM data through robot clients. We are using ODBC Actual Technology drivers for that purpose.
Our backups are running on 30 minutes intervals. The size of all the dbs is around 20Gb spread in around 80 dbs. Most of our dbs are one db with one table. The reason for that is an easy and fast recovery in case of a damaged file.

We have recently moved from FM11 to FM13, primary because of the memory leak we have noticed on our old server that was leading to server crashes. As soon as we hit 8.5Gb of Real Mem usage for the fmserverd on the Activity Console, there was a good chance we would get a crash, followed with verification data and corruption on some of the opened dbs.
We have worked with couple of people from Filemaker on that issue, but they were not able to find the reason for the memory leak. The workaround was to restart the server as soon as it hits the 8.4Gb real mem value.
We thought that with FM13, 64 bit app, and all the new features, our crashes will be part of the history, but we are not that lucky.
Our current FM server is on Mac Pro, 3.2. GHz Quad Core Intel Xeon, with 32 GB DDR3 RAM, and the HD is 500Gb SSD, with 300Gb available space.
Our clients are on many different types of computers, but no Go devices yet. OS on those devices are ranging from 10.8.x to 10.10.x
The memory leak that we had from FM11 is probably still with us in FM13, just a bit more intense maybe. Because of that, we still are restarting the server on timely basis, just at different real memory amount. When we restart the FM 13 server the start up real mem reading is 8.5 Gb. We have noticed that the server was crushing when we reach 14.5Gb and up. To reach 14.5Gb threshold, it takes around 7-10 days. For FM11 to reach the 8.4Gb threshold it took 17-21 days, so we have to restart the server more frequently than before.
One very interesting fact is that in lot of our crashes, when we open the console it is unresponsive. One can try to disconnect clients, close dbs, stop the server/web publishing engine, but nothing is responding or happening visually. Then we need to restart the computer itself.
Another thing I have noticed is that when we get a crash, on the console often it says that a backup is still running, and the time of the backup’s start is maybe 20+ minuets prior to the crash. Usually our backups finish in 2-3 minutes, so it saying that it is still running after 20 minutes is maybe a lead to some of the reasons for the crashes.
I am concerned that the system is not able to un-pause the dbs while it’s backing them up, and that is putting it in some kind of loop, that leads to a crash.
I can submit the log files to the interested parties. I have two logs’ locations:
/library/Filemaker Server/Logs - 4.22Gb total, due to one of the log files stderr being 4.08Gb and it's being constantly updated.
/Library/Logs - where the DiagnosticsReports folder is located (1.1 Mb total)
I have tried to open the stderr file, but none of the app I have tried are opening it (Text Edit, Word, TextWrangler).

Please, let me know which log files will be beneficial for me to upload?

I apologize if this issue was already discussed in some other thread. I have found some threads about FM Server crashing, but was not able to see the same reasons as in my case.

Thank you.
Pajo

Expected result

Solution to prevent crashes

Actual result

None yet

Workaround

none so far