Fedora Salt Master - New installation

DustinB3403

@scottalanmiller said in Fedora Salt Master - New installation:

Do a...

netstat -tulpn

netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      799/sshd
tcp        0      0 master-ip:4505     0.0.0.0:*               LISTEN      1004/python2.7
tcp        0      0 master-ip:4506     0.0.0.0:*               LISTEN      1010/python2.7
tcp6       0      0 :::22                   :::*                    LISTEN      799/sshd
tcp6       0      0 :::9090                 :::*                    LISTEN      1/systemd
udp        0      0 0.0.0.0:68              0.0.0.0:*                           914/dhclient
udp        0      0 127.0.0.1:323           0.0.0.0:*                           755/chronyd
udp6       0      0 ::1:323                 :::*                                755/chronyd

scottalanmiller

@DustinB3403 said in Fedora Salt Master - New installation:

@scottalanmiller said in Fedora Salt Master - New installation:

Do a...

netstat -tulpn

netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      799/sshd
tcp        0      0 master-ip:4505     0.0.0.0:*               LISTEN      1004/python2.7
tcp        0      0 master-ip:4506     0.0.0.0:*               LISTEN      1010/python2.7
tcp6       0      0 :::22                   :::*                    LISTEN      799/sshd
tcp6       0      0 :::9090                 :::*                    LISTEN      1/systemd
udp        0      0 0.0.0.0:68              0.0.0.0:*                           914/dhclient
udp        0      0 127.0.0.1:323           0.0.0.0:*                           755/chronyd
udp6       0      0 ::1:323                 :::*                                755/chronyd

There you go, a Python process is using those ports.

DustinB3403

@scottalanmiller said in Fedora Salt Master - New installation:

There you go, a Python process is using those ports.

That's the salt-master. .

DustinB3403

Full systemctl status as it was cut off before showing that Python is being used by this service.

 systemctl status salt-master.service
● salt-master.service - The Salt Master Server
   Loaded: loaded (/usr/lib/systemd/system/salt-master.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2019-05-08 15:57:31 EDT; 5min ago
	 Docs: man:salt-master(1)
		   file:///usr/share/doc/salt/html/contents.html
		   https://docs.saltstack.com/en/latest/contents.html
 Main PID: 995 (salt-master)
	Tasks: 32 (limit: 2350)
   Memory: 454.6M
   CGroup: /system.slice/salt-master.service
		   ├─ 995 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─1002 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─1004 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─1005 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─1008 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─1009 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─1010 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─1011 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─1018 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─1019 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─1020 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─1021 /usr/bin/python2.7 /usr/bin/salt-master
		   └─1025 /usr/bin/python2.7 /usr/bin/salt-master

May 08 15:57:30 localhost.localdomain systemd[1]: Starting The Salt Master Server...
May 08 15:57:30 localhost.localdomain salt-master[995]: /usr/lib/python2.7/site-packages/salt/scripts.py:102: DeprecationWarning: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 w>
May 08 15:57:31 localhost.localdomain systemd[1]: Started The Salt Master Server.

coliver

@DustinB3403 said in Fedora Salt Master - New installation:

Full systemctl status as it was cut off before showing that Python is being used by this service.

systemctl status salt-master.service
● salt-master.service - The Salt Master Server
Loaded: loaded (/usr/lib/systemd/system/salt-master.service; enabled; vendor preset: disabled)
Active: active (running) since Wed 2019-05-08 15:57:31 EDT; 5min ago
Docs: man:salt-master(1)
file:///usr/share/doc/salt/html/contents.html
https://docs.saltstack.com/en/latest/contents.html
Main PID: 995 (salt-master)
Tasks: 32 (limit: 2350)
Memory: 454.6M
CGroup: /system.slice/salt-master.service
├─ 995 /usr/bin/python2.7 /usr/bin/salt-master
├─1002 /usr/bin/python2.7 /usr/bin/salt-master
├─1004 /usr/bin/python2.7 /usr/bin/salt-master
├─1005 /usr/bin/python2.7 /usr/bin/salt-master
├─1008 /usr/bin/python2.7 /usr/bin/salt-master
├─1009 /usr/bin/python2.7 /usr/bin/salt-master
├─1010 /usr/bin/python2.7 /usr/bin/salt-master
├─1011 /usr/bin/python2.7 /usr/bin/salt-master
├─1018 /usr/bin/python2.7 /usr/bin/salt-master
├─1019 /usr/bin/python2.7 /usr/bin/salt-master
├─1020 /usr/bin/python2.7 /usr/bin/salt-master
├─1021 /usr/bin/python2.7 /usr/bin/salt-master
└─1025 /usr/bin/python2.7 /usr/bin/salt-master

May 08 15:57:30 localhost.localdomain systemd[1]: Starting The Salt Master Server...
May 08 15:57:30 localhost.localdomain salt-master[995]: /usr/lib/python2.7/site-packages/salt/scripts.py:102: DeprecationWarning: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 w>
May 08 15:57:31 localhost.localdomain systemd[1]: Started The Salt Master Server.

So the Salt-Master is running.

DustinB3403

@coliver yea, it's running, but it can't use the ports it automatically setup to run. .

Which is why I'm asking if I've missed something. .

scottalanmiller

@DustinB3403 said in Fedora Salt Master - New installation:

@coliver yea, it's running, but it can't use the ports it automatically setup to run. .

Which is why I'm asking if I've missed something. .

We just determined that it is using those ports.

scottalanmiller

So what we now know is that the Salt Master is up and running, and it is using the right ports. So the original issue has been determined to not exist.

So now that Salt is doing what it is supposed to be doing, is the issue something like that you can't connect from the client? If so, maybe you've not opened the firewall yet?

Curtis

I was just thinking check the firewall...

DustinB3403

I just did pkill salt-master and am checking salt-master --log-level=debug and have more output this time.

scottalanmiller

@DustinB3403 said in Fedora Salt Master - New installation:

I just did pkill salt-master and am checking salt-master --log-level=debug and have more output this time.

What are you looking for? I thought that it was fixed.

JaredBusch

@scottalanmiller said in Fedora Salt Master - New installation:

@DustinB3403 said in Fedora Salt Master - New installation:

@coliver yea, it's running, but it can't use the ports it automatically setup to run. .

Which is why I'm asking if I've missed something. .

We just determined that it is using those ports.

It wasn't starting because SELinux. it seems as it started after he disabled it.

So now it is starting, but if it is still not tlaking, then that is likely because of the firewall as @Curtis said

scottalanmiller

@DustinB3403 said in Fedora Salt Master - New installation:

I just did pkill salt-master and am checking salt-master --log-level=debug and have more output this time.

You should not be killing or starting Salt manually. Use systemctl to start and stop. I think that's what is confusing you.

scottalanmiller

@JaredBusch said in Fedora Salt Master - New installation:

@scottalanmiller said in Fedora Salt Master - New installation:

@DustinB3403 said in Fedora Salt Master - New installation:

@coliver yea, it's running, but it can't use the ports it automatically setup to run. .

Which is why I'm asking if I've missed something. .

We just determined that it is using those ports.

It wasn't starting because SELinux. it seems as it started after he disabled it.

So now it is starting, but if it is still not tlaking, then that is likely because of the firewall as @Curtis said

Okay, so it is all fixed now?

DustinB3403

@scottalanmiller said in Fedora Salt Master - New installation:

@DustinB3403 said in Fedora Salt Master - New installation:

I just did pkill salt-master and am checking salt-master --log-level=debug and have more output this time.

You should not be killing or starting Salt manually. Use systemctl to start and stop. I think that's what is confusing you.

I just tried that as a step because the dev github recommended it for another person who is having the same exact issue.

Literally brand new install of Fed 30 Server.

@JaredBusch, the service was always running, the ports have never worked.

scottalanmiller

@DustinB3403 said in Fedora Salt Master - New installation:

@JaredBusch, the service was always running, the ports have never worked.

Something is wrong there. The thing that is determing that the "ports do not work" was based on trying to start the Salt Master, but it was already running. That tells us that the ports were working.

Other than that, why do you feel that the ports are not working?

DustinB3403

firewall-cmd --list-ports
4505/tcp 4506/tcp

DustinB3403

 firewall-cmd --list-ports
4505/tcp 4506/tcp
[root@localhost ~]# systemctl stop salt-master
[root@localhost ~]# systemctl stop salt-master.service
[root@localhost ~]# systemctl start salt-master.service
[root@localhost ~]# systemctl status salt-master.service
● salt-master.service - The Salt Master Server
   Loaded: loaded (/usr/lib/systemd/system/salt-master.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2019-05-08 16:31:42 EDT; 5s ago
	 Docs: man:salt-master(1)
		   file:///usr/share/doc/salt/html/contents.html
		   https://docs.saltstack.com/en/latest/contents.html
 Main PID: 3121 (salt-master)
	Tasks: 27 (limit: 2350)
   Memory: 387.5M
   CGroup: /system.slice/salt-master.service
		   ├─3121 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─3128 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─3130 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─3131 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─3134 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─3135 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─3136 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─3137 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─3139 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─3145 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─3146 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─3147 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─3150 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─3333 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─3334 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─3335 /usr/bin/python2.7 /usr/bin/salt-master
		   ├─3336 /usr/bin/python2.7 /usr/bin/salt-master
		   └─3339 /usr/bin/python2.7 /usr/bin/salt-master

May 08 16:31:41 localhost.localdomain systemd[1]: Starting The Salt Master Server...
May 08 16:31:41 localhost.localdomain salt-master[3121]: /usr/lib/python2.7/site-packages/salt/scripts.py:102: DeprecationWarning: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 >
May 08 16:31:42 localhost.localdomain systemd[1]: Started The Salt Master Server.
[root@localhost ~]# salt-master --log-level=debug
/usr/lib/python2.7/site-packages/salt/scripts.py:102: DeprecationWarning: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date.  Salt will drop support for Python 2.7 in the Sodium release or later.
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Configuration file path: /etc/salt/master
[WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
[INFO    ] Setting up the Salt Master
[WARNING ] Unable to bind socket master-ip:4505, error: [Errno 98] Address already in use; Is there another salt-master running?
[INFO    ] The Salt Master is shut down
[DEBUG   ] Stopping the multiprocessing logging queue listener
[DEBUG   ] closing multiprocessing queue
[DEBUG   ] joining multiprocessing queue thread
[DEBUG   ] Stopped the multiprocessing logging queue listener
The salt master is shutdown. The ports are not available to bind

DustinB3403

firewall-cmd --list-all
FedoraServer (active)
  target: default
  icmp-block-inversion: no
  interfaces: enp0s3
  sources:
  services: cockpit dhcpv6-client ssh
  ports: 4505/tcp 4506/tcp
  protocols:
  masquerade: no
  forward-ports:
  source-ports:
  icmp-blocks:
  rich rules:

scottalanmiller

Do a netstat -tulpn again. Anytime you see that those ports are in use, that means that Salt-Master is running. Not sure what is starting it, but there is no question that it is being started somewhere. Only one copy can bind to the ports.