Wialon Local is a server product, and to use it successfully, you need not only knowledge of Wialon algorithms but also the work of a server administrator. In this article, we’ve compiled the most common issues that Wialon Local server administrators have to address, as well as the instructions for solving them.
All actions described below should be performed by a qualified Linux server specialist. For your convenience, we’ve compiled a list of administering tasks in the user guide.
Administration system errors
When logging in to the administration system, you may encounter various errors. Based on the error code and name, choose one of the following solutions.
The site can't be reached / Unable to connect / Page not found
The site unavailability and the output of errors with such text may be related to the state of nginx.
First, you need to make sure that nginx is working:
An example of the output of responses to commands when the service is working correctly:
If there are syntax errors in the nginx configuration or nginx doesn’t work, you should solve the problem at the level of specialists who maintain the server.
404 Not Found
If the administration system is unavailable and returns the "404 Not Found" error, it’s most probably that the Wialon Local service was installed incorrectly, and the files needed for the service operation are simply missing.
In such a situation, you should:
Check if there is an "install" folder in the /home/wialon/wlocal directory.
If there is no such folder, the Wialon Local installation likely failed. If the folder exists, try to restart the Wialon Local process:
- Try to reinstall Wialon Local from the ISO image, following all the necessary requirements:
- You should carry out the installation in the BIOS mode, and for UEFI — in the Legacy mode;
- When installing the OS, you cannot skip the step with specifying the network connection settings;
- There should be no other network restrictions on the server that don’t allow downloading the appropriate packages for service operation.
If the result has not changed after reinstalling Wialon Local, please contact firstname.lastname@example.org to install the service manually.
502 Bad Gateway
The "502 Bad Gateway" error when accessing the administration system may be related to the status of processes connected with Wialon Local operation.
An example of the expected list of active processes from the first command in a working service:
If an empty response is received for the specified commands or the cron.js process from the first command is missing, the Wialon Local processes are not running. Try to start the service again:
If the service cannot start, you should:
Check the version of packages required for Wialon Local operation using the following commands
Responses to the first two commands must meet the Wialon Local version requirements specified in the documentation:
Wialon Local versions Debian versions Node.js versions 1504, 1604, 1704 8 (Jessie) 0.10.x 1704, 1804, 1904 9 (Stretch) 6.x 1904, 2004, 2104 10 (Buster) 10.x
If the Node.js version doesn’t match the required version, you should update it.
An empty response to the third and/or fourth command means there is no npm package and its forever module. To install them, consult the instructions in the documentation.
Analyze the Wialon Local service logs in /home/wialon/wlocal/logs/lcm/ and, if there are errors related to Node.js, reinstall the Wialon Local components required for correct operation, according to the documentation. An example of similar errors:
Look carefully at the output of the installation results to avoid overlooking errors that may occur during the update.
If the administration system is still unavailable after the steps taken, please write to email@example.com.
The "502 Bad Gateway" error while executing the report
If you were redirected to a page with a "502 Bad Gateway" error when executing a report in Wialon, or it is observed in the browser console when executing a report, it’s most probably the requested report is trying to download resources to which you have limited access.
This can happen when you execute a report that uses road speed limits related to Gurtam Maps. In this case, you should check if you have the necessary access to Gurtam Maps resources: lic.gurtam.com 18611-18614
If you use other maps (Yandex, Google), you should open appropriate access to their servers.
Fleetrun/Nimbus authorization issues
If you cannot log in to Fleetrun or Nimbus applications using the password you have for the wialon user, but at the same time you can log in to the monitoring site, you should check whether the application requirements are met.
- Availability of a separate DNS;
- Availability of a flespi token in the admin panel;
- A corresponding service is activated in the account;
- Full chain of SSL certificates for DNS that is used for the application (the application must work over HTTPS);
- For Nimbus: availability of connected Gurtam Maps;
- Direct access to:
mqtt.flespi.io 8883 (SSL) or 1883 (non-SSL)
If all requirements are met, but the app authorization issue remains, please contact firstname.lastname@example.org.
Increase of the sync cache size
When using the Wialon Local backup module, you may encounter a serious increase in the synchronization cache files with the backup server, which can result in the disk space overflow on the main server. An example of such a file name: /home/wialon/wlocal/storage/md/sync.cache.1983335884. This often occurs due to network problems between the main and backup servers, differences in the speed of data recording to disks, or because the communication channel speed is not enough to transfer the incoming data volume.
First of all, you should check if there are any network limitations between the backup and main servers, and conduct a basic check of the backup server availability over the standard 32001 port, for example, via telnet from the main Wialon Local server:
In this case, the 32001 port is the standard port specified in the administration system of the main server. This port is used for subsequent data synchronization. It can be changed, so carefully check if the corresponding ports are available on the backup and main servers between which you’ve set synchronization.
You should also analyze the backup module logs in the /home/wl_storage_backup/logs/trace.log file for errors. Particularly, logs may contain errors of the following type:
Such errors indicate the access issues between servers.
If the backup server is unavailable for a long time, or if the synchronization cache files take up too much disk space, you should stop the backup module on the backup server using the command:
After that, you need to solve the access issues between servers and delete the backup copy of the database in the /home/wl_storage_backup/storage directory.
Next, stop the Wialon service in the main server administration system, then execute the "service wlocal stop" command to stop the administration system site, and only then delete the critically overflowed synchronization cache files, for example, by using the command:
When manually deleting the synchronization cache files, keep in mind that the necessary files are located in different folders of the /home/wialon/wlocal/storage directory and have names in the following format: sync.cache.1983335884, where numbers at the end may change.
After completing all the procedures, the synchronization process is launched in two stages: first, Wialon Local is enabled on the main server, then on the backup one. To do this, use the following commands:
A "sync finished" entry in the /home/wialon/wlocal/logs/trace.log file indicates the end of the synchronization process on the backup server. Also, for additional verification, you can compare the statistics on messages on the main server in the /home/wialon/wlocal/storage/ms/msgs_stats.txt file and on the backup server in the /home/wl_storage_backup/storage/ms/msgs_stats.txt file — with complete synchronization, the numbers must correspond.
Checking the modem connection to Wialon Local
In case of any difficulties with connecting the modem to Wialon Local, you should analyze logs in /home/wialon/wlocal/logs
SMS transmission to be sent via an SMPP gateway or a GSM modem is logged in trace.log of the Wialon service. Communication with the gateway or GSM device, as well as SMS sending, is logged in /home/wialon/wlocal/logs/ with the smpp_device_* / gsm_device_* names.
Examples of logs:
After creating and enabling the SMPP gateway, a separate smpp_device _*.log file is created, in which all connection errors are recorded, and the fact of connection is logged.
All interactions with the gateway will also be displayed in trace.log.
For example, if there is no connection to the SMPP gateway, the specified logs will contain messages of the following format:
Interaction with GSM and network modems will be recorded only in trace.log until the first connection is established.
An example of entries in the log when there is no connection to the modem:
To diagnose the connection, always check the availability of a remote host or gateway and also that the modem is mounted at the address indicated in the admin panel and responds to commands (for example, via minicom).
If SMS messages are sent from the Wialon service but don’t reach the final destination according to logs, you should solve the problem on the recipient’s side or the modem side. If there is no connection to the modem or remote gateway, you should do the same: such issues should be addressed at your network, modem, or gateway level. In any other situation, please contact email@example.com.
Transferring Wialon Local to a new server
This can become necessary if there is no backup module at the moment of transition to another physical server. By following the instructions below, you will avoid issues with such a transition.
Transfer (migration) process of the Wialon Local service should be performed in the order described below:
Inform your personal manager about the date of transfer.
- Deploy Wialon Local on the new server from ISO image, strictly following the official instructions up to step 5 inclusive (i.e., do not log in to the admin panel).
- Stop the Wialon service in the administration system of the old server, and then execute the "service wlocal stop" command to stop the site of the administration system itself.
- Log in to the administration system of the new server, wait for the installation of the main modules required for Wialon Local operation.
- Start Wialon on the new server and make sure it works stably.
- Stop the Wialon service in the administration system of the new server, and then execute the "service wlocal stop" command to stop the administration system site.
Copy the contents of the /home/wialon/wlocal/storage directory from the old server to the new one into the same directory. Previously, delete the contents of the /storage directory on the new server. After the transfer, execute the following command on the new server:
- Start the Wialon Local service again (the service wlocal start command) and Wialon in the administration system on the new server to make sure that the database substitution was successful and the service works correctly.
If the backup module from the Wialon Local ISO image has already been installed on the server intended for the migration, then the migration instructions are the same as in the case of using the backup server as the main one. The procedure is described in the documentation.
Corrupted database files
If Wialon cannot start after an emergency stopping or a server crash, referring to corrupted database files, you can see the following entry in the trace.log file:
If the Wialon service continues to work, and the log contains the "Fatal error, run database recovery" entry, you should immediately stop Wialon (the service and the administration system site) and start recovery procedures.
First of all, if you are using a backup server, you should check your backup database files on the backup server in the /home/wl_storage_backup/storage/md and home/wl_storage_backup/storage/pd directories using the db_verfify script, which should be created in the /home/wialon/wlocal directory. Its contents are shown below. The script must be executed strictly from this very directory (home/wialon/wlocal) using the ./db_verify.sh command.
If you suppose that some specific db-files were damaged, you can substitute the name of a specific database file instead of *.db in the last line, for example, m-00000001.db. Then the line will look like this:
An example of the result of script execution:
If no errors are found as a result of verification, you have 2 options:
- Before performing any manipulations with database substitution, you should stop Wialon in the administration system on the main server and the Wialon Local service itself (service wlocal stop), and also stop the backup service on the backup server (service wlbackup stop). After that, replace the database on the main server (the /home/wialon/wlocal/storage directory), the database on the backup server (the /home/wl_storage_backup/storage directory) and start Wialon.
- Use the backup server as the main one according to the Restoring from Failure instructions.
If the db files on the backup server are also damaged, try to delete the backup database and restart the synchronization process. As soon as it finishes, check the copied database using the db_verify script and, if successful, you can substitute it on the main server. The database should be substituted from the /home/wl_storage_backup directory on the backup server instead of the /home/wialon/wlocal/storage directory on the main one. At the same time, it’s recommended to save the damaged database of the main server in a separate location and not delete it until you can start Wialon without errors.
If the solution with using the backup server didn’t work, you can try to use standard utilities to restore separate files manually. We also recommend that you run the db_verify script again and localize all broken db files before further actions. Let's consider this option using the /home/wialon/wlocal/storage/md/m-00000007.db file as an example (if there are other damaged files, you should do the same).
Stop the Wialon service in the administration system, and then execute the "service wlocal stop" command to stop the administration system site. After that, run the following commands in the /home/wialon/wlocal directory:
After that, you can start the database manipulations (scripts below are already in Wialon, and you just need to execute them in the /home/wialon/wlocal directory):
Then, in the /home/wialon/wlocal/storage/md directory, rename the new and old database file using the following commands:
These are standard database utilities that will re-create your db file.
Next, start Wialon and check trace.log for errors. Then run the m-00000007.db file and other broken files again through the db_verify script when the service is stopped.
If none of the above recommendations helped you, write to firstname.lastname@example.org and describe all the actions you performed, as well as the issues you encountered while restoring the database files.
Mail delivery issues
You may face a situation when emails from Wialon do not reach the recipients. Such an issue may occur in some of the services. For example, sending emails works only from the Wialon Local administration system but not from Wialon itself.
There are two mechanisms for sending email notifications in Wialon Local. Knowing their characteristics can help diagnose potential issues.
Notifications from the administration system are delivered to the administrator's mail specified in the settings (see the Mail system section on the System tab). They operate on the following priority:
- Sending emails using the SMTP server specified in the administration system;
- Sending emails via postfix if the SMTP settings are not set or if the email cannot be transmitted to the specified mail gateway.
Logs indicating the method and status of sending are located in the /home/wialon/wlocal/logs/lcm.log.* file.
When using your own SMTP server, the send logs should be analyzed on the side of the SMTP itself where it is deployed. In the case of using postfix, the logs can be viewed in /var/mail/ or /var/log/mail.log.*
It should also be noted that Wialon Local supports the use of other mail services besides postfix since the request to send an email is sent to localhost. Thus, if you decide, for example, to use exim instead of postfix, you should make sure that it can handle all incoming mail requests from the server's localhost address.
Starting with Wialon Local 2104, you specify not only the login for authorization in your mail gateway but also the sender's name in the Login field for SMTP server settings in the administration system. If only the Login field is filled out of all the SMTP server settings, then the request to send the email will go to postfix under the specified login. If postifix does not have a user with such a login, then the email will not be sent. Therefore, if you do not use your own SMTP server, then we recommend that you leave all the fields related to it empty.
Emails that are sent when jobs and notifications are completed operate on the following priority:
- Sending using the SMTP server specified in the billing plan (please note that the address of the SMTP server in the billing plan should be indicated along with the protocol, for example, "smtps://smtp.gmail.com");
- If SMTP is not set in the billing plan, then the email will be sent using the SMTP specified in the administration system;
- If none of the SMTP addresses is specified, the email will be sent via postfix.
The main difference from the previous mechanism is that if an email cannot be sent using one of the specified SMTP addresses, then it will not be further transmitted to another SMTP address or to postfix and such an email will simply not be sent in the end. If the email sending fails, you need to make the necessary adjustments to be able to keep using the desired email tool.
Logs of the service the message was sent to as well as its sending status can be found in the /home/wialon/wlocal/logs/trace.log file using the "email" filter. See the final logs of sending via the SMTP server directly at the SMTP server. Find the logs of sending via postfix at /var/mail/ or /var/log/mail.log.*
Following this mechanism, it is also possible to use your own mail service instead of postfix if it is similarly bound to the server's localhost address.
Troubleshooting and configuration of sending mail via SMTP or postfix should be handled by the technical specialist who maintains the Wialon Local server. Our technical support does not include diagnostics and mail system configuration services.
High RAM usage
When working with Wialon Local, you may notice that the service constantly consumes almost all the amount of RAM available on the server. This practice is quite expected, since the Wialon service caches almost all the available RAM, but actively uses only some of it.
Wialon Local caches almost all the available RAM for its work regardless of the amount on the server. The Wialon service caches all RAM for processing requests and tasks, for quick access to pages, and to avoid possible delays and issues associated with access denials due to RAM being used by other applications on the server. Cached RAM is managed by a load balancer that redistributes cached RAM to WIalon or other applications on the server, but the priority is given to Wialon by default.
You can control the use of memory in the administration system on the RAM chart. By hovering over each section of the chart, you can see a record of how much memory is currently actively used and how much is cached.
If there is not enough RAM for the correct operation of the Wialon service, check out the /home/wialon/wlocal/logs/trace.log file to see the following entries:
In such cases, you should check if the server meets the system requirements for the available number of objects. You should also check for the presence of services or applications that could be actively using RAM on the server at the time such an entry appeared and analyze the situation.
nginx configuration issues
All Wialon Local service websites operate using the nginx web server. Sometimes when updating or reinstalling the nginx web server, websites are no longer available for one reason or another. In such cases, you should return to the initial nginx configuration that was on the server after installation from the official Wialon Local image.
When installing Wialon Local from the official image, configuration files for websites are automatically created on the server. Sometimes they have to be edited due to the technical requirements of the server. All responsibility for configuring and setting nginx up lies strictly on the technical specialists who maintain the Wialon Local server.
For further work, it helps to understand what configuration files and settings are present on the server immediately after installing Wialon Local. All the necessary configuration files will be located in the /etc/nginx/conf.d/ directory and their standard appearance after installation is described below.
lcm.conf is the administration system configuration file.
cms.conf is the temporary CMS Manager configuration file.
webmonitor.conf is the temporary Wialon Web configuration file.
local.conf is the main configuration file that reflects all the settings of the sites specified in the administration system.
This file is a symbolic link to the file that is automatically generated based on the websites’ settings in the administration system and is located at /home/wialon/wlocal/nginx/conf/local.conf.
This configuration file is strictly forbidden to edit, we also do not recommend unlinking the symbolic link. All the necessary adjustments for your Wialon sites are recommended to be made in the administration system. If you use your own configuration file instead of the automatically generated local.conf, the Wialon Local server administrator is fully responsible for the functionality of the websites.
If for some reason the local.conf symlink is unlinked, it should be reverted with the command:
In the first three configuration files, the <your_IP> parameter in the server_name line indicates the local IP address of your Wialon Local server. The standard configuration files will be linked to it. This IP address can be changed.
It should be understood that the webmonitor.conf and cms.conf configuration files are temporary only because they are used for temporary access to Wialon sites via IP address and port. They are also used for initial service performance checks before creating separate DNS records for websites. As soon as you create a separate DNS entry for these websites in the administration system, these files will no longer be relevant and we recommend deleting them. Up to this point, you can access these websites at the following addresses:
- Wialon Local administration system: http://<your_IP> or via the loopback address of the server itself http://localhost:8080
- CMS Manager site: http://<your_IP>:8024 or via the loopback address of the server itself http://localhost:8024
- Wialon Web site: http://<your_IP>:8025 or via the loopback address of the server itself http://localhost:8025
All the necessary web server logs and errors can be found in /var/log/nginx/
nginx security improvements
Depending on the security and network requirements of the server, additional nginx configuration may be required. Below, you can find the most common examples and recommendations on how to supplement a web server configuration to increase security.
Extend support for TLS protocol versions at /etc/nginx/common/default.conf:
Set the maximum validity period for SSL certificates at /etc/nginx/common/default.conf:
Change the active list of encryptions at /etc/nginx/common/default.conf (the entire list of encryptions can be found here):
Manually add an SSL certificate for the administration system website at /etc/nginx/conf.d/lcm.conf:
In this case, you should set the absolute path to the certificate and its private key in the parameters above.
These settings are recommendations and are not mandatory. We do not provide technical support for setting up and configuring nginx.