While changing the architecture around a web application, a new Varnish process with a new VCL was fired up. After recent changes inside the VCL, the Varnish process needed to be restarted for testing purposes.
But after the restart, the process was gone.
root@varnish:~# systemctl restart varnish-new
root@varnish:~# systemctl status varnish-new
- varnish-new.service - Varnish HTTP accelerator
Loaded: loaded (/etc/systemd/system/varnish-new.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Wed 2021-08-04 11:32:28 CEST; 4s ago
Docs: https://www.varnish-cache.org/docs/4.1/
man:varnishd
Process: 3842635 ExecStart=/usr/sbin/varnishd -j unix,user=vcache -n varnish-new -F -a :6083 -T localhost:6084 -f /etc/varnish/new.vcl -S /etc/varnish/secret -s malloc,4096m (code=exited, status=2)
Main PID: 3842635 (code=exited, status=2)
Aug 04 11:32:28 onl-lb01-t varnishd[3842635]: Error:
Aug 04 11:32:28 onl-lb01-t varnishd[3842635]: Message from VCC-compiler:
Aug 04 11:32:28 onl-lb01-t varnishd[3842635]: Symbol not found.
Aug 04 11:32:28 onl-lb01-t varnishd[3842635]: ('Builtin' Line 32 Pos 1)
Aug 04 11:32:28 onl-lb01-t varnishd[3842635]: vcl 4.0;
Aug 04 11:32:28 onl-lb01-t varnishd[3842635]: ###-----
Aug 04 11:32:28 onl-lb01-t varnishd[3842635]: Running VCC-compiler failed, exited with 2
Aug 04 11:32:28 onl-lb01-t varnishd[3842635]: VCL compilation failed
Aug 04 11:32:28 onl-lb01-t systemd[1]: varnish-new.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Aug 04 11:32:28 onl-lb01-t systemd[1]: varnish-new.service: Failed with result 'exit-code'.
The systemctl status output helps to shift the focus on the VCL itself, as the VCC-compiler analyzes the VCL in the backgorund.
The VCL error can also be spotted directly using the varnishd daemon/program:
root@varnish:~# varnishd -C -f /etc/varnish/new.vcl
Message from VCC-compiler:
Symbol not found.
('Builtin' Line 32 Pos 1)
vcl 4.0;
###-----
Running VCC-compiler failed, exited with 2
VCL compilation failed
The output from VCC compiler is kind of cryptic: Symbol not found is not really helpful. At least "line 32" is shown as indication where to spot an error.
But line 32 is absolutely OK, no errors can be spotted there:
30 if (req.method == "PURGE") {
31 # Only for authorized requests
32 if (req.http.X-Purge-Auth == "secret123") {
33 return (purge);
34 } else {
35 return (synth(405, "Not allowed"));
36 }
37 }
Note: The exact same PURGE condition was defined in other VCL files, without resulting in an error.
A quick research online resulted in finding a very interesting hint on serverfault:
After entering a closing curly bracket, it worked
Taking a look at the last few lines of the VCL seems to confirm this:
root@varnish:~# tail -n 20 /etc/varnish/new.vcl
req.http.User-Agent ~ "(?i)portalmmm" ||
req.http.User-Agent ~ "(?i)proxinet" ||
req.http.User-Agent ~ "(?i)windows\ ?ce" ||
req.http.User-Agent ~ "(?i)winwap" ||
req.http.User-Agent ~ "(?i)eudoraweb" ||
req.http.User-Agent ~ "(?i)htc" ||
req.http.User-Agent ~ "(?i)240x320" ||
req.http.User-Agent ~ "(?i)avantgo") {
set req.http.X-UA-Device = "mobile";
}
}
# if (req.http.X-UA-Device == "desktop") {
# if (req.http.User-Agent ~ "; Trident/" || req.http.User-Agent ~ "; MSIE ") {
# set req.http.X-UA-Device-IE = "ie";
# } else if (req.http.User-Agent ~ " Edge/" || req.http.User-Agent ~ "Edge" || req.http.User-Agent ~ "Edg/") {
# set req.http.X-UA-Device-IE = "edge";
# }
# }
# }
The last if condition was commented/disabled, but it seems that the final closing curly bracket (which closes the sub function) was also commented by mistake.
After fixing this mistake...
root@varnish:~# tail -n 5 /etc/varnish/new.vcl
# }
# }
}
... the VCC compiler did not show any error anymore and the Varnish instance could be restarted without a problem:
root@varnish:~# systemctl restart varnish-new
root@varnish:~# systemctl status varnish-new
- varnish-new.service - Varnish HTTP accelerator
Loaded: loaded (/etc/systemd/system/varnish-new.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2021-08-04 11:36:42 CEST; 1s ago
Docs: https://www.varnish-cache.org/docs/4.1/
man:varnishd
Main PID: 3846002 (varnishd)
Tasks: 220 (limit: 9451)
Memory: 11.5M
CGroup: /system.slice/varnish-new.service
|-3846002 /usr/sbin/varnishd -j unix,user=vcache -n varnish-new -F -a :6083 -T localhost:6084 -f /etc/varnish/new.vcl -S /etc/varnish/secret -s malloc,4096m
|-3846035 /usr/sbin/varnishd -j unix,user=vcache -n varnish-new -F -a :6083 -T localhost:6084 -f /etc/varnish/new.vcl -S /etc/varnish/secret -s malloc,4096m
Aug 04 11:36:42 onl-lb01-t systemd[1]: Started Varnish HTTP accelerator.
Aug 04 11:36:43 onl-lb01-t varnishd[3846002]: Debug: Version: varnish-plus-6.0.7r3 revision 43d3792752799e34b9717ed87ac5c4274c62cad6
Aug 04 11:36:43 onl-lb01-t varnishd[3846002]: Version: varnish-plus-6.0.7r3 revision 43d3792752799e34b9717ed87ac5c4274c62cad6
Aug 04 11:36:43 onl-lb01-t varnishd[3846002]: Debug: Platform: Linux,5.4.0-66-generic,x86_64,-junix,-smalloc,-sdefault,-hcritbit
Aug 04 11:36:43 onl-lb01-t varnishd[3846002]: Platform: Linux,5.4.0-66-generic,x86_64,-junix,-smalloc,-sdefault,-hcritbit
Aug 04 11:36:43 onl-lb01-t varnishd[3846002]: Debug: Child (3846035) Started
Aug 04 11:36:43 onl-lb01-t varnishd[3846002]: Child (3846035) Started
Aug 04 11:36:43 onl-lb01-t varnishd[3846002]: Info: Child (3846035) said Child starts
Aug 04 11:36:43 onl-lb01-t varnishd[3846002]: Child (3846035) said Child starts
Usually the VCC compiler throws back very specific errors, e.g. wrong format (string), condition error, missing semicolon, etc. But the error "Symbol not found" is surprisingly non-specific. This can therefore only point to a "top-level" problem in the VCL, meaning a (sub) function not correctly closed (or opened) so that the compiler was not even able to read the VCL parts correctly.
No comments yet.
AWS Android Ansible Apache Apple Atlassian BSD Backup Bash Bluecoat CMS Chef Cloud Coding Consul Containers CouchDB DB DNS Database Databases Docker ELK Elasticsearch Filebeat FreeBSD Galera Git GlusterFS Grafana Graphics HAProxy HTML Hacks Hardware Icinga Influx Internet Java KVM Kibana Kodi Kubernetes LVM LXC Linux Logstash Mac Macintosh Mail MariaDB Minio MongoDB Monitoring Multimedia MySQL NFS Nagios Network Nginx OSSEC OTRS Office OpenSearch PGSQL PHP Perl Personal PostgreSQL Postgres PowerDNS Proxmox Proxy Python Rancher Rant Redis Roundcube SSL Samba Seafile Security Shell SmartOS Solaris Surveillance Systemd TLS Tomcat Ubuntu Unix VMWare VMware Varnish Virtualization Windows Wireless Wordpress Wyse ZFS Zoneminder