INTERFACING MODEMS TO VAX COMPUTERS
David W. Deley
(Revised November, 1990)
The following is a discussion on interfacing modems to VAX RS-232
computer ports when the port is set to MODEM/HANGUP.
Pin 7 is signal ground and is always connected. Pins 2 and 3 are
transmit and receive and are always connected. Pin 20 (DTR) is
controlled by the computer and monitored by the modem. This pin is used
by the computer to control the modem. When Pin 20 is high the modem
should be ready to answer any incoming calls. When pin 20 goes low the
modem should hang up and not answer anymore incoming calls.
The computer will pulse pin 20 low when it wants the modem to hang
up. There are two cases when the computer will do this. The first is
when a user logs out. When a user logs out the computer will pulse pin
20 low telling the modem to hang up because the user is through. The
second is when someone calls in and then doesn't log on. When the modem
answers an incoming call, the computer starts a 30 second timer. If no
one has logged in after 30 seconds, the computer will pulse pin 20 low
telling the modem to hang up because no one has logged in.
Pins 5, 6, and 8 are controlled by the modem and monitored by the
computer. The computer uses these pins to determine what the modem is
doing. In order for the computer to successfully control the modem, it
must be able to distinguish between four possible states:
1. The modem is on hook and no one is logged in. The computer holds
pin 20 high telling the modem that it is ready to receive any incoming
2. The modem is off hook but no one is logged in. The computer starts
a timer and pulses pin 20 low after 30 seconds, telling the modem to
hang up because no one has logged in.
3. The modem is off hook and a user is logged in. The computer holds
pin 20 high and talks to the user.
4. A user is logged in and the modem goes on hook. The computer
politely logs out the user so the next person who dials in won't get
connected to his process. The computer also goes through the formality
of pulsing pin 20 low.
Problems arise when the computer thinks the modem is in one state when
it really is in another. For instance, the modem may be on hook waiting
for an incoming call, while the computer thinks the modem is off hook
answering a call. In this case, the computer will continually cycle its
30 second timer, and pulse pin 20 low every thirty seconds. If a user
calls in and the modem answers, the computer may tell the modem to hang
up before the user has a chance to log in.
Floating lines can also cause intermittent problems. If a pin is left
disconnected so that it is neither high nor low, then the computer may
think it is high one day and low the next, creating problems which seem
to come and go without explanation.
It is unknown exactly what causes the computer to think one thing and
not another (see VMS I/O User's Reference Manual, chapter Terminal
Driver, section Terminal Driver Features, subsection Dial-Up Support),
and this is where the fun comes in. Luckily, there do seem to be some
conditions under which the computer appears to behave in a consistent
If pins 5, 6, and 8 are all low the computer thinks the modem is on
hook. In this case the computer will hold pin 20 (DTR) high telling the
modem that it should answer any incoming calls.
Various combinations of pins 5, 6, and 8 going high make the computer
think that the modem has answered an incoming call. If pins 5, 6, and 8
all go high the computer definitely thinks this, and it starts its 30
second timer. If at the end of 30 seconds no one has logged in, then
the computer pulses pin 20 low telling the modem to hang up.
With a user logged in and pins 5, 6, and 8 all high, the computer
behaves properly and does not tell the modem to hang up until the user
has logged out.
If a user is logged in and pins 5, 6, and 8 are all high, then various
combinations of pins 5, 6, and 8 going low will cause the computer to
think the modem has gone on hook and the computer will abruptly log the
user out. Various other combinations of pins 5, 6, and 8 going low will
cause the computer to log the user out after a short delay. If pins 5, 6,
and 8 all go low, then the computer definitely thinks the modem has gone
on hook and immediately logs the user out.
INTERFACING WITH THE RACAL-VADIC 2400PA AUTO-DIAL MODEM
We connected pins 7(GND), 2(transmit), 3(receive), and 20 (DTR)
with out problem. Pin 20 worked in hanging up the modem when it went
low. We then connected pins 5, 6, and 8 straight through. This didn't
work because when the modem was on hook it held pin 5 low and pins 6 and
8 high. This caused the computer to continually cycle through its 30
second timer routine and there were problems dialing in. We then
connected pin 5 from the modem to pins 5, 6, and 8 at the computer.
This seemed to work. Pin 5 was low when the modem was on hook, and it
went high after the modem had answered an incoming call and made
connection with the remote modem. Then the next day it didn't work
anymore. We discovered from reading the modem manual that the modem
doesn't assert pin 5 until it sees pin 4 as high, and we weren't
connecting pin 4. There was an optional setting in the modem's menu
that would have cured this problem, but in order to make a cable that
would be independent of this menu setting, we decided to also pass pin
4. The final cable passed pins 2,3,4,5,7,20 with pins 5,6,8 bridged at
the computer end. (This turned out to be incorrect).
It turned out that passing pin 4 did not make operation of the
modem independent of it's menu settings. With pin 4 passed the modem
always held pin 5 high which caused the computer to continually cycle
through its 30 second timeout sequence. The 2400PA modem MUST have
option 21 (CTS control) set to 2 (CTS follows CXR) in order for the
above cable to work properly. But since we are now setting modem
options, we might as well set option 2 (DSR control) to 3 (DSR normal),
and option 3 (CXR control) to 1 (CXR normal). With these settings the
modem can usually be connected with a straight cable passing pins
2,3,5,6,7,8, and 20.
CHAPETER TWO: INTERFACING A RIXON MODEM
Rixon modems could be connected directly to the computer port with a
straight through ribbon cable if options 4 and 5 in the modem menu are
disabled. Option 4 asks if DSR pin 6 should always be on, and option
5 asks if CTS pin 5 should always be on. If these options are enabled
then the computer will never know when a person hangs up the phone
without logging out because the modem will never drop these pins low.
By disabling these options the modem then works correctly, and the
computer is happy. Unfortunately, whenever the modem looses power it
always resets these two options. Thus the straight forward method of
interfacing the rixon to the computer is unacceptable.
It was then discovered that the rixon asserted pin 12 whenever a
connection was made, and pulled pin 12 low whenever the connection was
lost. This was exactly what we were looking for. We connected pin 12
from the rixon modem to pins 5, 6, and 8 on the computer port, and the
modem worked perfectly. The final cable passed pins 2, 3, 7, and 20
straight and pin 12 from the modem to pins 5, 6, and 8 on the computer.
According to the RS-232 standard, pin 12 is the high speed indicator and
is asserted whenever a 1200 bps connection is established. This means
that this cable probably won't work at 300 baud.
CHAPTER THREE: DIAL OUT MODEMS
All dial out modems connected to the computer should pass pin 20 so
the computer can hang up the modem. Also the modem port should be set
to MODEM/HANGUP. With this configuration, the computer will
automatically go through its hangup cycle whenever the port is
deallocated. This prevents people from connecting to the modem, dialing
out to a remote computer, breaking back to the local prompt and logging
out leaving the modem connected and logged in to a remote process.
CHAPETER FOUR: KILLING THE CYCLIC <LOGIN>
Some modems connected to computer ports can get into a situation
where the port is always in a <logon> state. The scenario goes
something like this. First a broadcasted message is sent to all the
ports by the system manager with a REPLY/ALL. This wakes up the modem
and it says "HI. HERE I AM". The computer then says, "Oh, someone's
there. I'll send him a USERNAME prompt." The modem responds saying
"WHAT?", and the computer says, "Good, now what's your password?" The
modem says "WHAT?" and the computer says "Sorry, logon failure." to
which the modem responds "WHAT?" and the computer says "Oh, someone's
there. I'll send him a USERNAME prompt..." and the cycle repeats
For dial-out only ports this problem can be solved by setting the
port to NOTYPE_AHEAD. This disables all logins on that port. It is
also recommended that the port be set to NOBROADCAST so that the modem
never gets broadcasted messages which can start the cycle or upset the
modem. For dial-in modems the problem can be solved by setting the port
to SYSPASSWORD. This disables all responses from the computer until the
SYSTEM_PASSWORD is entered. (Note that the SYSTEM_PASSWORD is not the
same as the password for the username SYSTEM). It is also possible that
simply setting the port to NOBROADCAST might be enough to prevent the
cycle from happening.
INTERFACING MODEMS TO DECSERVER (November 20, 1990)
Connecting DIAL-OUT modems to DECserver ports
Passing 2,3,7 for transmit, receive, ground, as always.
Passing 20 (computer to modem, tells modem to hangup) as always.
Passing 5,6,8 straight causes problems
DECserver will not send data to modem if 5 is low. 5 must be high
before modem can be contacted. (This apparently does not cause the
cyclic 30 second login timer to begin.) Some modems hold pin 5 high for
you. Some do not. Some modems will hold pin 5 high if they see pin 4
(from the DECserver) high. Or you could hardwire pin 5 to another pin
that is always high, if you can find one.
For dial-out modems, we hold pins 6 and 8 low and keep them low always.
Otherwise, the DECserver starts to interfere as follows. If pin 6 then
goes high, as a modem will when initiating a call, all subsequent data
from the modem is ignored by the DECserver. No data from the modem is
passed to the user until pin 8 also goes high. Thus the call progress
messages such as "DIALING...", "RINGING...", and "CONNECTION
ESTABLISHED" messages are not seen by the user, and the user must guess
at what is happening and when the connection has been completed. Also,
as if that wasn't enough, when you complete your call and the modem
hangs up, the DECserver will not talk to to the modem anymore until you
completely disconnect from the port the modem is connected to.
So for dial-out modems connected to DECservers, we pass 2,3,7,20, hold
pin 5 high, hold pins 6,8 low.
Connecting DIAL-IN modems to DECserver ports
Pass pins 2,3,7,20 as usual. The modem should hangup when the DECserver
pulls pin 20 low. Again the trick is to see what the modem does to pins
5,6,8, and figure out how the DECserver interprets pins 5,6,8. The
latter is documented in the DECserver system manager's manual. Pins
5,6,8 of the DECserver port eventually all need to be high before the
DECserver will allow a user to login. The DECserver should then be
notified if the modem hangs up by seeing either pins 6 or 8 go low, so
it can logout any process that is still logged in.
We ended up passing pin 8 from the modem to pins 5,6,8 of the DECserver.
Pin 8 stays low until the modem has answered an incoming call and fully
established a connection at which point it goes high, resulting in pins
5,6,8 of the DECserver all going high at once, allowing a user to login.
Other configurations including a straight pass of 5,6,8 would probably
work depending on what your modem does.