Archived Forum Post

Index of archived forum posts

Question:

Chilkat Ftp2 works with Stunnel.exe, but not without Stunnel.exe?

Oct 19 '15 at 09:58

If I use stunnel.exe (from stunnel.org), with the following stunnel config file, then Chilkat Ftp2 works and can establish data connections for uploading/downloading files. Here's the stunnel config file: 

client=yes


[My-FTP]
accept  = 2021
connect = my.ftpserver.com:2021
CAfile = my-ftp-servers-cert.pem
verify = 3
[0]
accept  = 4000
connect = my.ftpserver.com:2000
CAfile = my-ftp-servers-cert.pem
verify = 3
[1]
accept  = 4001
connect = my.ftpserver.com:2001
CAfile = my-ftp-servers-cert.pem
verify = 3
[2]
accept  = 4002
connect = my.ftpserver.com:2002
CAfile = my-ftp-servers-cert.pem
verify = 3


...


[20]
accept  = 4020
connect = my.ftpserver.com:2020
CAfile = my-ftp-servers-cert.pem
verify = 3

My application uses Chilkat Ftp2 and connects to localhost:2021 (which is the stunnel.exe). From my application's point of view, it is communicating with an FTP server without SSL/TLS, and it operates in passive mode for establishing data connections. The stunnel.exe provides the bi-directional SSL/TLS connections to the actual FTP server.

Also note:

  1. I cannot get FileZilla to work with this FTP server.
  2. If I do not use stunnel.exe, I cannot get Chilkat to work with FTPS using a direct SSL/TLS connection to this server. (I can establish the control/command connection w/ SSL/TLS using Chilkat, but it is the data connection that fails to complete.)

Answer

There are a number of things to explain. This is a very strange and unusual situation.

  1. The PEM file is unrelated to the problems. The PEM file is used by stunnel to verify that the remote server we're connecting to has and uses the certificate we expect in the PEM specified by CAfile in the config. The "verify=3" option in the config tells stunnel to verify that the remote server's SSL/TLS certificate is the one we expect. We can eliminate both the "CAfile" and "verify" lines from the config file, and everything would still work, but the server's cert would simply not be checked. In summary, the PEM file is not related to the data connection problems.
  2. stunnel is NOT a solution for the FTP protocol. See https://www.stunnel.org/faq.html In the particular case, stunnel is being used in combination with the pre-known behavior for this particular FTP server (in it's particular environment) and it is not a general solution.

So how can this possibly work with stunnel, but now with FileZilla, Chilkat, or any other robust and mature FTP client software?

To get the answer, we have to examine and understand exactly what is happening. Here's what's happening when a data connection (in passive mode) is established in this situation through stunnel:

  1. Chilkat (or FileZilla) sends a PASV command on the control connection to the server (via stunnel).
  2. stunnel forwards the PASV command to the FTP server via the TLS connection.
  3. The FTP server chooses a port and sends a PASV reply. Let's say it chooses port 4010. The server is saying "I'm listening at port 4010 and will be waiting for you to connect".
  4. The stunnel.exe forwards this reply back to app.
  5. The app (thinking that localhost:2021 is the actual FTP server), initiates a connection to localhost:4010.
  6. The stunnel.exe accepts the connection on port 4010 (because this is one of it's listen ports in the stunnel's config), but then initiates a connection to port 2010.
  7. Aha!!!! Here's the problem!!!! The FTP server told us that it's waiting for the connection on port 4010, but it's ACTUALLY waiting for the connection on port 2010!
  8. The FTP server accepts the connection on port 2010, and the data transfer happens.

It turns out that stunnel is really being used to "map" the passive port from one value to another. Port 4010 becomes 2010. Port 4003 becomes port 2003, etc. Stunnel is just being used to account for this highly unusual (and incorrect) behavior.

When the FTP server replies to a PASV command, it SHOULD be replying with the port at which it will be listening for the data connection. If it says "I'll be listening at port 4014", it cannot then listen at port 2014 and expect the FTP server to magically know this. This is why FileZilla, Chilkat, and all other perfectly good and mature FTP clients cannot handle this situation. It is because the server is not behaving properly. The "sane" FTP clients (FileZilla, Chilkat, etc.) are trying to connect to the port that was clearly indicated by the server. Sadly, the server is not actually listening at the port. It lied.

The solution is to fix the incorrect behavior of the server. Perhaps it isn't the FTP server itself, but perhaps it is a software or hardware (deep-inspection) firewall that is modifying the PASV reply. If so, this is crazy and I've never seen anything like it before. The only possible solution here is to fix the server side so that the server is actually listening on the port that it says it is..