This memo was created for the convenience of Wialon Local administrator. We tried to gather here a brief list of the necessary accesses, peculiarity of some standard procedures (disk space control, working with logs and backups), as well as prohibited actions during the work and maintenance of Wialon Local.
lic.gurtam.com 31176 — license components check.
local-api.wialon.com 443 — scripts’ and modules’ updates.
lic.gurtam.com 18711 — Wialon LBS service.
lic.gurtam.com 18712 — Wialon mobile push notifications service.
lic.gurtam.com 18611-18616 — Gurtam Maps service.
https://distro.gurtam.com/maps/ — the old AVD maps storage.
https://api.telegram.org — Telegram notifications service.
https://acme-v02.api.letsencrypt.org/ — Let's Encrypt auto certificates (mandatory require the port 80 to be opened).
mqtt.flespi.io 8883 (SSL) or 1883 (not SSL) — MQTT broker.
app-local.wialon.com port 443 — to access NimBus, Fleetrun, Hecterra, Logistics, and other Wialon-based apps.
Default port range for Wialon hardware in iptables:
-A INPUT -p udp -m state --state NEW -m udp --dport 20100:30000 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 20100:30000 -j ACCEPT
Port 1935 or 19350 depending on the device type.
Port 8083 — to get files from the video server.
Wialon Local is installed to /home/wialon/wlocal by default.
All the Wialon Local logs are located in /home/wialon/wlocal/logs/ by default. They are stored and rotated for the last 10 days (logrotate).
Conditions for forced database synchronization anew:
This scheme shows general components of Wialon Local and the interaction between them. It can be useful for system administrators for a basic understanding of Wialon Local architecture, and can also be used for participating in tenders.
Wialon Local is a server product, and to use it successfully, you need not only knowledge of Wialon algorithms but also the work of a server administrator. In this article, we’ve compiled the most common issues that Wialon Local server administrators have to address, as well as the instructions for solving them.
All actions described below should be performed by a qualified Linux server specialist. For your convenience, we’ve compiled a list of administering tasks in the user guide.
When logging in to the administration system, you may encounter various errors. Based on the error code and name, choose one of the following solutions.
The site unavailability and the output of errors with such text may be related to the state of nginx.
First, you need to make sure that nginx is working:
service nginx restart service nginx status #(checking the service status) nginx -t #(checking the syntax of files if the service doesn’t start)
An example of the output of responses to commands when the service is working correctly:
nginx.service - A high performance web server and a reverse proxy server Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: en Active: active (running) since Tue 2021-08-03 13:46:09 MSK; 2min 56s ago nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful
If there are syntax errors in the nginx configuration or nginx doesn’t work, you should solve the problem at the level of specialists who maintain the server.
If the administration system is unavailable and returns the "404 Not Found" error, it’s most probably that the Wialon Local service was installed incorrectly, and the files needed for the service operation are simply missing.
In such a situation, you should:
Check if there is an "install" folder in the /home/wialon/wlocal directory.
If there is no such folder, the Wialon Local installation likely failed. If the folder exists, try to restart the Wialon Local process:
service wlocal restart
If the result has not changed after reinstalling Wialon Local, please contact email@example.com to install the service manually.
The "502 Bad Gateway" error when accessing the administration system may be related to the status of processes connected with Wialon Local operation.
ps aux | grep -v grep | grep node top | grep wialon
An example of the expected list of active processes from the first command in a working service:
wialon 701 0.0 2.3 591208 47884 ? Ssl 12:27 0:00 /usr/bin/node /usr/lib/node_modules/forever/bin/monitor cron.js wialon 713 1.3 36.4 1491260 745596 ? Sl 12:27 0:49 /usr/bin/node /home/wialon/wlocal/cron.js
If an empty response is received for the specified commands or the cron.js process from the first command is missing, the Wialon Local processes are not running. Try to start the service again:
service wlocal restart service wlocal status #(checking the start)
If the service cannot start, you should:
Check the version of packages required for Wialon Local operation using the following commands
node -v nodejs -v npm -v npm -g list | grep forever
Responses to the first two commands must meet the Wialon Local version requirements specified in the documentation:
|Wialon Local versions||Debian versions||Node.js versions|
|1504, 1604, 1704||8 (Jessie)||0.10.x|
|1704, 1804, 1904||9 (Stretch)||6.x|
|1904, 2004, 2104||10 (Buster)||10.x|
If the Node.js version doesn’t match the required version, you should update it.
An empty response to the third and/or fourth command means there is no npm package and its forever module. To install them, consult the instructions in the documentation (2004/2104 and 1704/1804/1904).
Analyze the Wialon Local service logs in /home/wialon/wlocal/logs/lcm/ and, if there are errors related to Node.js, reinstall the Wialon Local components required for correct operation, according to the documentation. An example of similar errors:
Error: Most middleware (like json) is no longer bundled with Express and must be installed separately ...
Look carefully at the output of the installation results to avoid overlooking errors that may occur during the update.
If the administration system is still unavailable after the steps taken, please write to firstname.lastname@example.org.
If you were redirected to a page with a "502 Bad Gateway" error when executing a report in Wialon, or it is observed in the browser console when executing a report, it’s most probably the requested report is trying to download resources to which you have limited access.
This can happen when you execute a report that uses road speed limits related to Gurtam Maps. In this case, you should check if you have the necessary access to Gurtam Maps resources: lic.gurtam.com 18611-18614
If you use other maps (Yandex, Google), you should open appropriate access to their servers.
If you cannot log in to Fleetrun or Nimbus applications using the password you have for the wialon user, but at the same time you can log in to the monitoring site, you should check whether the application requirements are met.
Requirements for Fleetrun and Nimbus operation in Wialon Local are described in the documentation:
mqtt.flespi.io 8883 (SSL) or 1883 (non-SSL)
If all requirements are met, but the app authorization issue remains, please contact email@example.com.
When using the Wialon Local backup module, you may encounter a serious increase in the synchronization cache files with the backup server, which can result in the disk space overflow on the main server. An example of such a file name: /home/wialon/wlocal/storage/md/sync.cache.1983335884. This often occurs due to network problems between the main and backup servers, differences in the speed of data recording to disks, or because the communication channel speed is not enough to transfer the incoming data volume.
First of all, you should check if there are any network limitations between the backup and main servers, and conduct a basic check of the backup server availability over the standard 32001 port, for example, via telnet from the main Wialon Local server:
In this case, the 32001 port is the standard port specified in the administration system of the main server. This port is used for subsequent data synchronization. It can be changed, so carefully check if the corresponding ports are available on the backup and main servers between which you’ve set synchronization.
You should also analyze the backup module logs in the /home/wl_storage_backup/logs/trace.log file for errors. Particularly, logs may contain errors of the following type:
2021/05/24 17:19:50:007: [INF:73907700] net_session::bg_task(2167042, '192.168.0.15:32001'): pipe not connected 2021/05/24 17:19:50:007: [INF:4CA1E700] messages_sync_server::sync_thread('192.168.0.15:32001:******'): invalid reply from synchronization server
Such errors indicate the access issues between servers.
If the backup server is unavailable for a long time, or if the synchronization cache files take up too much disk space, you should stop the backup module on the backup server using the command:
service wlbackup stop
After that, you need to solve the access issues between servers and delete the backup copy of the database in the /home/wl_storage_backup/storage directory.
Next, stop the Wialon service in the main server administration system, then execute the "service wlocal stop" command to stop the administration system site, and only then delete the critically overflowed synchronization cache files, for example, by using the command:
find /home/wialon/wlocal/storage -name sync.cache.* -type f -delete
When manually deleting the synchronization cache files, keep in mind that the necessary files are located in different folders of the /home/wialon/wlocal/storage directory and have names in the following format: sync.cache.1983335884, where numbers at the end may change.
After completing all the procedures, the synchronization process is launched in two stages: first, Wialon Local is enabled on the main server, then on the backup one. To do this, use the following commands:
On the main: service wlocal start On the backup: service wlbackup start
A "sync finished" entry in the /home/wialon/wlocal/logs/trace.log file indicates the end of the synchronization process on the backup server. Also, for additional verification, you can compare the statistics on messages on the main server in the /home/wialon/wlocal/storage/ms/msgs_stats.txt file and on the backup server in the /home/wl_storage_backup/storage/ms/msgs_stats.txt file — with complete synchronization, the numbers must correspond.
In case of any difficulties with connecting the modem to Wialon Local, you should analyze logs in /home/wialon/wlocal/logs
SMS transmission to be sent via an SMPP gateway or a GSM modem is logged in trace.log of the Wialon service. Communication with the gateway or GSM device, as well as SMS sending, is logged in /home/wialon/wlocal/logs/ with the smpp_device_* / gsm_device_* names.
Examples of logs:
After creating and enabling the SMPP gateway, a separate smpp_device _*.log file is created, in which all connection errors are recorded, and the fact of connection is logged.
All interactions with the gateway will also be displayed in trace.log.
For example, if there is no connection to the SMPP gateway, the specified logs will contain messages of the following format:
2021/07/16 15:50:52:031: [INF:C2B98700] Can`t connect to SMPP server 2021/07/16 15:51:06:558: [INF:C2B98700] Can`t connect to SMPP server 2021/07/16 15:52:12:289: [INF:C2B98700] Can`t connect to SMPP server
2021/07/16 15:51:06:556: [INF:C2B98700] avl_gsm_device::begin_comm('Testing', 'd7d1eeb6c3361727b848012bf5a540d0'): 0 2021/07/16 15:51:06:558: [INF:C2B98700] Can`t connect to SMPP server
Interaction with GSM and network modems will be recorded only in trace.log until the first connection is established.
An example of entries in the log when there is no connection to the modem:
2021/07/16 15:55:16:332: [INF:C2B98700] avl_gsm_device::begin_comm('Testing', 'd7d1eeb6c3361727b848012bf5a540d0'): 0 2021/07/16 15:55:16:332: [INF:C2B98700] remote_gsm_device::start(88) And no more messages of communication with the modem.
2021/07/16 15:56:23:400: [INF:AADC7700] avl_gsm_device::begin_comm('Testing', 'd7d1eeb6c3361727b848012bf5a540d0'): 0 And no more messages of communication with the modem.
To diagnose the connection, always check the availability of a remote host or gateway and also that the modem is mounted at the address indicated in the admin panel and responds to commands (for example, via minicom).
If SMS messages are sent from the Wialon service but don’t reach the final destination according to logs, you should solve the problem on the recipient’s side or the modem side. If there is no connection to the modem or remote gateway, you should do the same: such issues should be addressed at your network, modem, or gateway level. In any other situation, please contact firstname.lastname@example.org.
This can become necessary if there is no backup module at the moment of transition to another physical server. By following the instructions below, you will avoid issues with such a transition.
Transfer (migration) process of the Wialon Local service should be performed in the order described below:
Inform your personal manager about the date of transfer.
Copy the contents of the /home/wialon/wlocal/storage directory from the old server to the new one into the same directory. Previously, delete the contents of the /storage directory on the new server. After the transfer, execute the following command on the new server:
chown -R wialon:wialon /home/wialon/
If the backup module from the Wialon Local ISO image has already been installed on the server intended for the migration, then the migration instructions are the same as in the case of using the backup server as the main one. The procedure is described in the documentation.
If Wialon cannot start after an emergency stopping or a server crash, referring to corrupted database files, you can see the following entry in the trace.log file:
2021/08/17 12:50:26:649: [INF:FAB26740] storage_service_db::open_databases: opening database environment... 2021/08/17 12:50:26:713: [INF:FAB26740] storage_service_db::open_databases: opening databases... 2021/08/17 12:50:26:714: [INF:FAB26740] storage_messages_env::open_environment: opening database environment (cache: size: 12104 MB, chunks: 1)... 2021/08/17 12:50:28:049: [INF:FAB26740] storage_messages_env::open_environment: recovering environment: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery 2021/08/17 12:50:28:841: [INF:FAB26740] adf_load_plugin('storage_server'). Error loading plugin. Error initializing storage subsystem as server. 2021/08/17 12:50:28:841: [INF:FAB26740] Error: couldn`t load ADF plugin: 'storage_server'.
If the Wialon service continues to work, and the log contains the "Fatal error, run database recovery" entry, you should immediately stop Wialon (the service and the administration system site) and start recovery procedures.
First of all, if you are using a backup server, you should check your backup database files on the backup server in the /home/wl_storage_backup/storage/md and home/wl_storage_backup/storage/pd directories using the db_verfify script, which should be created in the /home/wialon/wlocal directory. Its contents are shown below. The script must be executed strictly from this very directory (home/wialon/wlocal) using the ./db_verify.sh command.
#! /bin/sh export PATH="./modules/adf/bin:./modules/dep/bin:$PATH" export LD_LIBRARY_PATH="./modules/adf/lib:./modules/dep/lib:$LD_LIBRARY_PATH" export ADF_SCRIPT_PATH="modules/adf_scripts/scripts" export TCL_LIBRARY="./modules/dep/lib/tcl8.4" ulimit -c 0 db_verify /home/wialon/wlocal/storage/pd/*.db db_verify /home/wialon/wlocal/storage/md/*.db
If you suppose that some specific db-files were damaged, you can substitute the name of a specific database file instead of *.db in the last line, for example, m-00000001.db. Then the line will look like this:
An example of the result of script execution:
Successful verification: BDB5105 Verification of /home/wialon/wlocal/storage/md/m-00000001.db succeeded. BDB5105 Verification of /home/wialon/wlocal/storage/md/m-00000002.db succeeded. BDB5105 Verification of /home/wialon/wlocal/storage/md/m-00000003.db succeeded. DB file verification failed: BDB5105 Verification of /home/wialon/wlocal/storage/md/m-00000003.db failed. BDB5105 Verification of /home/wialon/wlocal/storage/md/m-00000004.db failed.
If no errors are found as a result of verification, you have 2 options:
If the db files on the backup server are also damaged, try to delete the backup database and restart the synchronization process. As soon as it finishes, check the copied database using the db_verify script and, if successful, you can substitute it on the main server. The database should be substituted from the /home/wl_storage_backup directory on the backup server instead of the /home/wialon/wlocal/storage directory on the main one. At the same time, it’s recommended to save the damaged database of the main server in a separate location and not delete it until you can start Wialon without errors.
If the solution with using the backup server didn’t work, you can try to use standard utilities to restore separate files manually. We also recommend that you run the db_verify script again and localize all broken db files before further actions. Let's consider this option using the /home/wialon/wlocal/storage/md/m-00000007.db file as an example (if there are other damaged files, you should do the same).
Stop the Wialon service in the administration system, and then execute the "service wlocal stop" command to stop the administration system site. After that, run the following commands in the /home/wialon/wlocal directory:
export PATH="./modules/adf/bin:./modules/dep/bin:$PATH" export LD_LIBRARY_PATH="./modules/adf/lib:./modules/dep/lib:$LD_LIBRARY_PATH" export ADF_SCRIPT_PATH="modules/adf_scripts/scripts" export TCL_LIBRARY="./modules/dep/lib/tcl8.4" ulimit -c 0
After that, you can start the database manipulations (scripts below are already in Wialon, and you just need to execute them in the /home/wialon/wlocal directory):
db_dump -r /home/wialon/wlocal/storage/md/m-00000007.db | db_load /home/wialon/wlocal/storage/md/m-00000007.db.new
Then, in the /home/wialon/wlocal/storage/md directory, rename the new and old database file using the following commands:
mv m-00000007.db m-00000007.db.old mv m-00000007.db.new m-00000007.db
These are standard database utilities that will re-create your db file.
Next, start Wialon and check trace.log for errors. Then run the m-00000007.db file and other broken files again through the db_verify script when the service is stopped.
If none of the above recommendations helped you, write to email@example.com and describe all the actions you performed, as well as the issues you encountered while restoring the database files.