View Issue Details

IDProjectCategoryView StatusLast Update
0006619Kali LinuxKali Package Bugpublic2020-08-07 15:57
ReporterRoseDeSable Assigned Tosbrun  
PrioritynormalSeveritymajorReproducibilityalways
Status resolvedResolutionfixed 
Product Versionkali-dev 
Summary0006619: gvm: at start time ospd writes an error message in the log - it seems to exist difficulties with the redis-server
Description

Hi,
this message comes, if ospd-openvas starts:

Message from syslogd@<my-system> at Aug 4 06:30:35 ...
Traceback (most recent call last):0000012 File "/usr/bin/ospd-openvas", line 11, in <module>0000012 load_entry_point('ospd-openvas==1.0.1', 'console_scripts', 'ospd-openvas')()0000012 File "/usr/lib/python3/dist-packages/ospd_openvas/daemon.py", line 1670, in main#012 daemon_main('OSPD - openvas', OSPDopenvas)0000012 File "/usr/lib/python3/dist-packages/ospd/main.py", line 157, in main#012 daemon.init()0000012 File "/usr/lib/python3/dist-packages/ospd_openvas/daemon.py", line 298, in init#012 self.load_vts()0000012 File "/usr/lib/python3/dist-packages/ospd_openvas/daemon.py", line 582, in load_vts#012 oids = dict(self.nvti.get_oids())0000012 File "/usr/lib/python3/dist-packages/ospd_openvas/nvticache.py", line 139, in get_oids#012 return self._openvas_db.get_elem_pattern_by_index('filename:*')0000012 File "/usr/lib/python3/dist-packages/ospd_openvas/db.py", line 385, in get_elem_pattern_by_index#012 elem_list.append([item, ctx.lindex(item, index)])0000012 File "/usr/lib/python3/dist-packages/redis/client.py", line 1657, in lindex#012 return self.execute_command('LINDEX', name, index)0000012 File "/usr/lib/python3/dist-packages/redis/client.py", line 839, in execute_command#012 return self.parse_response(conn, command_name, **options)0000012 File "/usr/lib/python3/dist-packages/redis/client.py", line 853, in parse_response#012 response = connection.read_response()0000012 File "/usr/lib/python3/dist-packages/redis/connection.py", line 718, in read_response#012 raise response#012redis.exceptions.ResponseError: WRONGTYPE Operation against a key holding the wrong kind of value

Further I see, that some components for running gvm cannot open their own pid-files

the redis-server: redis-server.service: Can't open PID file /run/redis/redis-server.pid

the gvmd-service: gvmd.service: Can't open PID file /run/gvm/gvmd.pid (yet?) after s...

In some discussions in the internet, I find the note, that the reason for the message "Traceback ..." seems to be an incompatibility of the versions of the components. Here are the versions and releases of my installation:

ospd-openvas --version
OSP Server for openvas: 1.0.1
OSP: 1.2
OSPd: 2.0.1

Copyright (C) 2014, 2015, 2018, 2019 Greenbone Networks GmbH
License GPLv2+: GNU GPL version 2 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Best Regards
Rose

Activities

sbrun

sbrun

2020-08-04 08:06

manager   ~0013178

I think you have issue with redis configuration. You mentioned this:
redis-server.service: Can't open PID file /run/redis/redis-server.pid

But we are not using redis-server.service with openvas-scanner but [email protected]
Can you check and install the latest version of openvas-scanner (should be 7.0.1-1kali2)?

You must have this in the file /etc/openvas/openvas.conf:
db_address = /var/run/redis-openvas/redis-server.sock

And where do you find the information that it "seems to be an incompatibility of the versions"?

fresnel

fresnel

2020-08-04 11:55

reporter   ~0013180

Hi,
I have the same exact problem,

"raise response#012redis.exceptions.ResponseError: WRONGTYPE Operation against a key holding the wrong kind of value"

As a further detail, I can say that the [email protected] cannot create the pid file but it seems that is only temporary.

Aug 04 13:07:33 ovscan4 systemd[1]: [email protected]: Can't open PID file /run/redis-openvas/redis-server.pid (yet?) after start: Operation not permitted
Aug 04 13:07:33 ovscan4 systemd[1]: Started Advanced key-value store (openvas).

here is the output og gvm-check-setup:

gvm-check-setup 1.0.0
Test completeness and readiness of GVM-11
Step 1: Checking OpenVAS (Scanner)...
OK: OpenVAS Scanner is present in version 7.0.1.
OK: Server CA Certificate is present as /var/lib/gvm/CA/servercert.pem.
OK: redis-server is present.
OK: scanner (db_address setting) is configured properly using the redis-server socket: /var/run/redis-openvas/redis-server.sock
OK: redis-server is running and listening on socket: /var/run/redis-openvas/redis-server.sock.
OK: redis-server configuration is OK and redis-server is running.
OK: NVT collection in /var/lib/openvas/plugins contains 60579 NVTs.
openvas-dump.rdb
OK: The redis database exists.
OK: OpenVAS Scanner is present in version 1.0.1.
Step 2: Checking GVMD Manager ...
OK: GVM Manager (gvmd) is present in version 9.0.1.
Step 3: Checking Certificates ...
OK: GVM client certificate is valid and present as /var/lib/gvm/CA/clientcert.pem.
OK: Your GVM certificate infrastructure passed validation.
Step 4: Checking data ...
OK: SCAP data found in /var/lib/gvm/scap-data.
Step 5: Checking user ...
OK: At least one user exists.
Step 6: Checking Greenbone Security Assistant (GSA) ...
Oops, secure memory pool already initialized
OK: Greenbone Security Assistant is present in version 9.0.1.
Step 7: Checking if GVM services are up and running ...
OK: gvmd service is active.
OK: ospd-openvas service is active.
OK: greenbone-security-assistant service is active.
Step 8: Checking GVM database ...
could not change directory to "/root": Permission denied
OK: portnames are in database.
Step 9: Checking few other requirements...
OK: nmap is present in version 9.0.1.
OK: ssh-keygen found, LSC credential generation for GNU/Linux targets is likely to work.
WARNING: Could not find makensis binary, LSC credential package generation for Microsoft Windows targets will not work.
SUGGEST: Install nsis.
OK: xsltproc found.
WARNING: Your password policy is empty.
SUGGEST: Edit the /etc/gvm/pwpolicy.conf file to set a password policy.

It seems like your GVM-11 installation is OK.

Everything seems fine here

Best Regards

sbrun

sbrun

2020-08-04 12:30

manager   ~0013181

Did you have openvas installed before or is it a fresh install of gvm ?
If you had openvas installed, did you remove the previous redis DB as indicated in MIGRATION-FROM-9-TO-11 and restart redis?

fresnel

fresnel

2020-08-04 12:41

reporter   ~0013183

I had a previous version of openvas installed, but I carefully folowed the guide.
In particular regarding the redis server i executed the following commands:

systemctl restart [email protected]
runuser -u _gvm -- redis-cli -s /var/run/redis-openvas/redis-server.sock flushall
systemctl restart [email protected]

fresnel

fresnel

2020-08-04 12:53

reporter   ~0013184

UPDATE:
executing gvm-check-setup returned this error:

Step 7: Checking if GVM services are up and running ...
Starting ospd-openvas service
Waiting for ospd-openvas service

    ERROR: ospd-openvas service didn't start.
    Please check journalctl -xe

The output of journalxctl -xe is:

-- The job identifier is 649722.
Aug 04 15:49:00 ovscan4 systemd[1]: Started OSPD OpenVAS.
-- Subject: A start job for unit ospd-openvas.service has finished successfully
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- A start job for unit ospd-openvas.service has finished successfully.
--
-- The job identifier is 649722.
Aug 04 15:49:01 ovscan4 ospd-openvas[2535893]: Traceback (most recent call last):
File "/usr/bin/ospd-openvas", line 11, in <module>
load_entry_point('ospd-openvas==1.0.1', 'console_scripts', 'ospd-openvas')()
File "/usr/lib/python3/dist-packages/ospd_openvas/daemon.py", line 1670, in main
daemon_main('OSPD - openvas', OSPDopenvas)
File "/usr/lib/python3/dist-packages/ospd/main.py", line 157, in main
daemon.init()
File "/usr/lib/python3/dist-packages/ospd_openvas/daemon.py", line 298, in init
self.load_vts()
File "/usr/lib/python3/dist-packages/ospd_openvas/daemon.py", line 582, in load_vts
oids = dict(self.nvti.get_oids())
File "/usr/lib/python3/dist-packages/ospd_openvas/nvticache.py", line 139, in get_oids
return self._openvas_db.get_elem_pattern_by_index('filename:*')
File "/usr/lib/python3/dist-packages/ospd_openvas/db.py", line 385, in get_elem_pattern_by_index
elem_list.append([item, ctx.lindex(item, index)])
File "/usr/lib/python3/dist-packages/redis/client.py", line 1657, in lindex
return self.execute_command('LINDEX', name, index)
File "/usr/lib/python3/dist-packages/redis/client.py", line 839, in execute_command
return self.parse_response(conn, command_name, **options)
File "/usr/lib/python3/dist-packages/redis/client.py", line 853, in parse_response
response = connection.read_response()
File "/usr/lib/python3/dist-packages/redis/connection.py", line 718, in read_response
raise response
redis.exceptions.ResponseError: WRONGTYPE Operation against a key holding the wrong kind of value
Aug 04 15:49:01 ovscan4 systemd[1]: ospd-openvas.service: Main process exited, code=exited, status=1/FAILURE
-- Subject: Unit process exited
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- An ExecStart= process belonging to unit ospd-openvas.service has exited.
--
-- The process' exit code is 'exited' and its exit status is 1.
Aug 04 15:49:01 ovscan4 systemd[1]: ospd-openvas.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- The unit ospd-openvas.service has entered the 'failed'

RoseDeSable

RoseDeSable

2020-08-04 13:27

reporter   ~0013185

Ok,

1) openvas was allready installed on my system

2) db_address = /var/run/redis-openvas/redis-server.sock is in the /etc/openvas/openvas.conf

3) Now I installed the update of the openvas-scanner from today

But I did not migrate the db ! The db must be a postgres-db or not ?

sbrun

sbrun

2020-08-04 13:41

manager   ~0013188

OK. If you already removed the old redis DB, I don't know what caused this issue.
Can you update your system to get the latest openvas-scanner (version 7.0.1-1kali2) and ospd-openvas (version 1.0.1-1kali1)
Remove again the redis DB
And just try to start the ospd-openvas.service?

RoseDeSable

RoseDeSable

2020-08-04 13:44

reporter   ~0013189

Ok,
I do it tomorrow, if I have time free.

Bye
Rose

RoseDeSable

RoseDeSable

2020-08-04 13:55

reporter   ~0013190

Ok,
openvas-scanner and ospd-openvas are both on the wished versions. The message traceback doesn't occur. But gvmd cannot open its pid-file. Ospd-openvas has no problems.

Bye
Rose

sbrun

sbrun

2020-08-04 13:55

manager   ~0013191

@RoseDeSable my answer was for @fresnel
GVM only support postgresql database now. You don't have to migrate your db if you don't want to keep your data.
Please read the MIGRATION-FROM-9-TO-11 file provided by the package gvm.

The redis DB is a different thing. Even if you don't migrate your old sqlite database to postgresql, you will have to remove
the old redis DB as explained in MIGRATION-FROM-9-TO-11.

sbrun

sbrun

2020-08-04 13:58

manager   ~0013192

@RoseDeSable Sorry, I answered at the same time as you
Good news if Ospd-openvas has no problems

fresnel

fresnel

2020-08-04 15:30

reporter   ~0013200

I think I'm already at the latest version

gvm:
Installed: 11.0.2~kali1
Candidate: 11.0.2~kali1
Version table:
*** 11.0.2~kali1 500
500 http://http.kali.org/kali kali-rolling/main amd64 Packages
100 /var/lib/dpkg/status

ospd-openvas:
Installed: 1.0.1-1kali1
Candidate: 1.0.1-1kali1
Version table:
*** 1.0.1-1kali1 500
500 http://http.kali.org/kali kali-rolling/main amd64 Packages
100 /var/lib/dpkg/status

and I removed again the redis DB but with no luck

UPDATE:
If I start ospd-openvas before gvmd It start well and the output of gvm-check-setup seems fine (I still have the trackback though)

Tomorrow I will try to perform a fresh installation and I'll let you know
In the meantime thanks for the help

fresnel

fresnel

2020-08-06 11:13

reporter   ~0013212

Hi @sbrun,
first of all thanks for your help, I performed an update of ospd-openvas right now and everything seems fine!
I don't have the trackback message anymore, and now I'm able to see all the NVTs
If you don't mind, I would keep this ticket open for a couple of hours just to check if it's all working well as it seems. Otherwise you can close it.
I'm curious of what was the issue: if you could tell me, it'll be very much appreciated

UPDATE: all works well but when I perform a scan it seems that user _gvm doesn't have permissions to execute nmap, in the scan results i have the following message:

ERROR: You requested a Nmap scan type which requires root privileges but scanner is running under an unprivileged user. Start scanner as root or use a different portrange to get this scan working.

I tried to add _gvm to the sudoers with this lines:

Defaults secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/gvm/sbin"
...
...
_gvm ALL = NOPASSWD: /opt/gvm/sbin/openvas

But with no luck
Do you have any hints?

_gvm ALL = NOPASSWD: /opt/gvm/sbin/gsad

sbrun

sbrun

2020-08-06 12:40

manager   ~0013213

@fresnel
I'm not sure what the issue was. Probably something wrong with the redis database but why the flushall didn't work? I don't know.
We changed the redis configuration to follow upstream recommendations: we no longer
save the DB redis on disk. Maybe that helps to fix your issue.

You should not need to add _gvm to sudoers. It's already added if you have the ospd-openvas version 1.0.1-1kali2: you must have the file /etc/sudoers.d/ospd-openvas.
And the path to the binary openvas in Kali is /usr/sbin/openvas (not /opt/gvm/sbin/openvas)

fresnel

fresnel

2020-08-06 13:08

reporter   ~0013215

@sbrun
Yes you are right.
Actually, I have the file /etc/sudoers.d/ospd-openvas
and it is correctly configured, in fact there is the line:

_gvm ALL = NOPASSWD: / usr / sbin / openvas

This makes it even more difficult for me to understand what's going on ...

sbrun

sbrun

2020-08-06 14:11

manager   ~0013216

I don't undestand either... You can try to check the log of openvas:
/var/log/gvm/openvas.log
Or restart the gvm services.

kali-bugreport

kali-bugreport

2020-08-06 20:26

reporter   ~0013223

Also check /var/log/gvm/ospd-openvas.log and try to run "sudo -u _gvm -n openvas -s" to test if the sudo setup is correct.

fresnel

fresnel

2020-08-07 08:54

reporter   ~0013231

Hi @sbrun and @kali-bugreport thanks for your help,
I have some good and bad news
The good one is that the server with permission issues is now working well and is able to perform scans correctly.
I don't know if it was relevant but I executed this commands:

groupadd pcap
usermod -a -G pcap _gvm
chgrp pcap /usr/sbin/tcpdump
chmod 750 /usr/sbin/tcpdump
sudo setcap cap_net_raw,cap_net_admin=eip /usr/sbin/tcpdump
sudo setcap cap_net_raw,cap_net_admin=eip /usr/bin/winexe
sudo setcap cap_net_raw,cap_net_admin=eip /usr/bin/openvas-nasl
sudo setcap cap_net_raw,cap_net_admin=eip /usr/bin/openvas-nasl-lint
sudo setcap cap_net_raw,cap_net_admin=eip /usr/bin/ospd-openvas

and I restarted the GVM services

The bad one is that on another server (identical to the other) I still have the redis trackback issue.

In this case an update wasn't enough, I repeatedly performed a flush of the redis DB with a restart but the issue persists.

fresnel

fresnel

2020-08-07 08:55

reporter   ~0013232

@kali-bugreport I don't have the ospd-openvas.log file, how can I enable it?

fresnel

fresnel

2020-08-07 09:26

reporter   ~0013233

@sbrun @kali-bugreport
Ok maybe I have some interesting infos.
If I flush the redis server and then I start the ospd-openvas.service, it starts fine and begins to reload NVTs

...

openvas: Reloaded 57600 of 61145 NVTs (94% / ETA: 00:02)

...

3943918 openvas: Reloaded 60050 of 61145 NVTs (98% / ETA: 00:00)

But when it reaches the 100%, the redis trackback appears and the service shuts down, tries to restart itself and every time it does it (a minute or so) the trackback message appears again

this is the ospd-openvas.service status:

ospd-openvas.service - OSPD OpenVAS
Loaded: loaded (/lib/systemd/system/ospd-openvas.service; disabled; vendor preset: disabled)
Active: activating (auto-restart) (Result: exit-code) since Fri 2020-08-07 11:23:41 CEST; 55s ago
Process: 3944403 ExecStart=/usr/bin/ospd-openvas --unix-socket=/run/ospd/ospd.sock --pid-file=/run/ospd/ospd-openvas.pid (code=exited, status=0/SUCCESS)
Main PID: 3944405 (code=exited, status=1/FAILURE)

fresnel

fresnel

2020-08-07 09:45

reporter   ~0013234

@sbrun @kali-bugreport

Another info: when i flush the server and ospd restarts, when it reaches 100% of NVTs reload I have the following error message:

OSPD - openvas: ERROR: (ospd_openvas.daemon) OpenVAS Scanner failed to load NVTs. Command '['openvas', '--update-vt-info']' died with <Signals.SIGPIPE: 13>.

I have to specify one thing: feed updates are made by root user, because I'm behind a proxy server and sudo doesn't pass the credentials, can it be a problem?

fresnel

fresnel

2020-08-07 10:28

reporter   ~0013235

@sbrun @kali-bugreport

Ok now it seems to work!!
I stopped completely gvm
then i performed a flush of the redis DB
then i updated the feed
and lastly restarted gvm

gvm-stop
sudo runuser -u _gvm -- redis-cli -s /var/run/redis-openvas/redis-server.sock flushall
systemctl restart [email protected]
gvm-feed-update
gvm-start

Please, leave this tk open until I check everything

sbrun

sbrun

2020-08-07 10:35

manager   ~0013236

OK. Thanks for the update.

fresnel

fresnel

2020-08-07 14:59

reporter   ~0013242

Hi,
the issue seems resolved, you can close the ticket now.
Thank you for your help

Issue History

Date Modified Username Field Change
2020-08-04 05:22 RoseDeSable New Issue
2020-08-04 08:06 sbrun Note Added: 0013178
2020-08-04 11:55 fresnel Note Added: 0013180
2020-08-04 12:30 sbrun Note Added: 0013181
2020-08-04 12:41 fresnel Note Added: 0013183
2020-08-04 12:53 fresnel Note Added: 0013184
2020-08-04 13:27 RoseDeSable Note Added: 0013185
2020-08-04 13:41 sbrun Note Added: 0013188
2020-08-04 13:44 RoseDeSable Note Added: 0013189
2020-08-04 13:55 RoseDeSable Note Added: 0013190
2020-08-04 13:55 sbrun Note Added: 0013191
2020-08-04 13:58 sbrun Note Added: 0013192
2020-08-04 15:30 fresnel Note Added: 0013200
2020-08-05 10:19 sbrun Assigned To => sbrun
2020-08-05 10:19 sbrun Status new => assigned
2020-08-06 11:13 fresnel Note Added: 0013212
2020-08-06 12:40 sbrun Note Added: 0013213
2020-08-06 13:08 fresnel Note Added: 0013215
2020-08-06 14:11 sbrun Note Added: 0013216
2020-08-06 20:26 kali-bugreport Note Added: 0013223
2020-08-07 08:54 fresnel Note Added: 0013231
2020-08-07 08:55 fresnel Note Added: 0013232
2020-08-07 09:26 fresnel Note Added: 0013233
2020-08-07 09:45 fresnel Note Added: 0013234
2020-08-07 10:28 fresnel Note Added: 0013235
2020-08-07 10:35 sbrun Note Added: 0013236
2020-08-07 14:59 fresnel Note Added: 0013242
2020-08-07 15:57 sbrun Status assigned => resolved
2020-08-07 15:57 sbrun Resolution open => fixed