Rational Developer for System z

IBM Rational Developer for System z


Answers to common host configuration and maintenance issues


SC14-7373-00
Note

Before using this information, be sure to read the general information under Notices IBM Rational Developer for System z.

First edition (June, 2012)

This edition applies to IBM Rational Developer for System z Version 8.5 (program number 5724-T07) and to all subsequent releases and modifications until otherwise indicated in new editions.

Order publications by phone or fax. IBM Software Manufacturing Solutions takes publication orders between 8:30 a.m. and 7:00 p.m. eastern standard time (EST). The phone number is (800) 879-2755. The fax number is (800) 445-9269. Faxes should be sent Attn: Publications, 3rd floor.

You can also order publications through your IBM representative or the IBM branch office serving your locality. Publications are not stocked at the address below.

IBM welcomes your comments. You can send your comments by mail to the following address:

IBM Corporation
Attn: Information Development Department 53NA
Building 501 P.O. Box 12195
Research Triangle Park NC 27709-2195.
USA

When you send information to IBM, you grant IBM a nonexclusive right to use or distribute the information in any way it believes appropriate without incurring any obligation to you.

Note to U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

Copyright International Business Machines Corporation 2012, 2012.
US Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

Contents

Figures
Tables
Answers to common host configuration and maintenance issues
Commonly encountered messages
Messages
rseserver.log
syslog
ELAXF* procedures
Client
Communication messages
rsecomm.log
rsedaemon.log
rseserver.log
syslog
Client
Security messages
R_ticketserv (IRRSPK) return codes
stderr.log
CA ACF2 / Top Secret (TSS)
JMON joblog
syslog
Hints and tips
Behavior changes
APPC no longer default
PassTicket setup required
Application security setup required
Multiple logons by a single userid
Log location
Server bind address
Host based property groups
Host based projects
Debugging and tracing host issues
FTP to IBM
SVC dump
APPC TP data
APPC trace
ISPF ISPVCALL trace
TCP/IP RESOLVER trace
TCP/IP and RESOLVER CTRACE
SAF trace
Show active z/OS UNIX processes
Show z/OS UNIX reason code
GSK SSL trace
Java SSL trace
Java method trace
CARMA memory leak trace
Installation and setup
FEKFIVPT invocation results in initialization failure
IVP test results in "CEE3201S operation exception" error
Starting the RSE daemon task fails with error message BPXM009I
FEKFIVPI failing with MSG: ISPZINT not found
fekfivpd when RSE started task is not active
fekfivpd with insufficient memory
Generating pass ticket results in ERROR ServerThread: 517: SafRc=8
RSED start with IVP parameter fails with Access Error
Host node name cannot be found
Developer for System z client fails to connect to host using SSL
RSEG1242 "Daemon failed to launch server on <host> using port <port>"
Access Error to IRRPTAUTH.FEKAPPL.* profile in PTKTDATA security class
Installation and setup for CARMA and Endevor®
Return code -3 when starting the CARMA server
ERROR com.ibm.carma.rse.server.miners.CARMAMiner: CARMA error occured RDZ - ENDEVOR error 500
CARMA server is not starting up in timely manner
Notices IBM Rational Developer for System z

Figures

  1. CRRZIO277E System ABEND E37-4 error message
  2. Error message RSEG1057
  3. RSEG1242 with connection refused
  4. RSEG1242 with server failure
  5. RSEG1242 with user failure
  6. RSEG1243 error message

Tables

  1. Security-related z/OS UNIX reason codes
  2. R_ticketserv (IRRSPK00) return codes

Answers to common host configuration and maintenance issues

Commonly encountered messages

You may see various error messages while running Rational® Developer for System z®. This section lists some common messages, their causes, and what you can do to correct the problem.

For explanations of the purposes of the different log files mentioned in this document, see the Rational Developer for System z Host Configuration Reference Guide (SC14-7290), chapter "Troubleshooting configuration problems", section "Log files".

Messages

rseserver.log

This section describes errors you might see in the rseserver.log.

server failure: error binding socket

 

   Message

ERROR RseDaemon: server failure: error binding socket

   Explanation

RSE server is not allowed to bind to a port. In this case, _RSE_PORTRANGE is probably defined, and the ports in this range are secured in TCPIP PROFILE, but with an incorrect filter. RSED* must be allowed to bind to these ports. Note: The * is important.

   Additional information

None.

getServerPort(): result=-1;No Access Authority (FEKAPPL)

 

Message

ERROR RseDaemon: getServerPort(): result=-1;No Access Authority (FEKAPPL)

Explanation

This problem can be caused by faulty security setup for the APPL class (as indicated in the message). But missing program control status for LE modules (see RACF® setup) can also result in this error, because Developer for System z is not able to load the code needed to do the security checks. In that case, the system log will show RACF messages like the following messages:

Additional information

None.

userid: User has insufficient permission to profile FEKAPPL in the APPL class, RACF return and reason code=_

 

Message

ERROR RseDaemon: userid: User has insufficient permission to profile FEKAPPL in the APPL class, RACF return and reason code=00d8

Explanation

This problem can be caused by faulty security setup for the APPL class (as indicated in the message), but it can also indicate a range of other problems.

The listed return code is the z/OS® UNIX reason code (errnojr), and is documented in the UNIX System Services Messages and Codes manual.

Location: http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/BPXZA8A0/3.3

The following table lists some of the possible reason codes:

Table 1. Security-related z/OS UNIX reason codes
00d8

JRNotServerAuthorized

The calling address space is not permitted to the BPX.SERVER facility class or the BPX.SERVER facility class is undefined and caller not a superuser (UID=0).

02af

JREnvDirty

The specified function is not supported in an address space where a load was done that is not program controlled.

Additional information

None.

syslog

This section describes errors you might see in the syslog.

USER COMPLETION CODE=4093

 

Message

USER COMPLETION CODE=4093 REASON CODE=00000090

Explanation

ABEND issued during initialization when errors were detected.

See Language Environment Run-Time Messages (SA22-7566), section "Language Environment® Abend Codes", code U4093 (X'FFD').

Location: http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/CEEA9190/SPTCS00020 (z/OS 1.10)

Reason codes

90x - Condition management for MVS™ could not be initialized

Explanation: This abend and reason code, issued during JMON startup, usually indicates that an OMVS RACF segment has not been defined for the user ID associated with the JMON started task.

Additional information: None.

ELAXF* procedures

IEF451I userid PLI STP0000 - ENDED BY CC 1016

 

Message

IEF451I IBMUSER1 PLI STP0000 - ENDED BY CC 1016

Explanation

For PL/I compiles, the PL/I listing data set (DD PLI.SYSPRINT) has the wrong characteristics. A valid combination is:

DSORG=PO,RECFM=VBA,LRECL=137,BLKSIZE=27998

Additional information

None.

Client

CRRZI0277E and CRRZI0282E

 

Message

CRRZI0277E System ABEND E37-4 CRRZI0282E A data set opened for output used all space available to or on the current volume, and no more volumes were available.

Explanation

In most cases, an E37-4 does not mean the volume is out of space, but rather that the target data set is out of extents. But Developer for System z shows the official MVS message text, so making the situation clear is difficult.

For the explanation of message IEC032I E37-4, see:

 http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/iea2m7a0/3.18

A data set opened for output used all space available to or on the current volume, and no more volumes were available. Change the JCL to specify more volumes.

  1. Not enough volumes were specified for the data set, through the SER, volume count, or REF sub-parameter of the VOLUME parameter of the DD statement. When all the volumes were filled, the program attempted to write another record.
  2. For a partitioned data set on a direct access volume or for a VIO data set, all space was filled when the program attempted to write another record. (A partitioned data set or a VIO data set can reside on only one volume with a maximum of 65535 tracks.)
  3. For a partitioned data set on a direct access volume, 16 extents had been used when the program attempted to write another record.

Figure 1. CRRZIO277E System ABEND E37-4 error message
CRRZIO277E System ABEND E37-4 error message

Additional information

None.

Communication messages

rsecomm.log

This section describes errors you might see in the rsecomm.log.

BufferUnderflowException

 

Message

ERROR MVSFileSystemMiner: java.nio.BufferUnderflowException 
  java.nio.BufferUnderflowException
  at java.nio.Buffer.nextGetIndex(Buffer.java:419) 
  at java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:351) 
  at com.ibm.fmi.server.dataprovider.fm.FMConnection.readSocketMessage
     (FMConnection.java:178)

Message on client

CRRZF6001E Error checking IBM® File Manager Version

Explanation

File Manager (FM) is not running APF-authorized. FM Version 10 does not have PTF UK54428 applied. The manual bypass is to copy module FMN0POPT from SFMNMOD1 to SFMNMODA.

(For this situation, Rational Developer for System z Version 7.6 client shows error message: Error checking IBM File Manager Version with CRRZF6001E.)

Additional information

None.

No lock daemon port

 

Message

ERROR FFSServerImpl: java.lang.RuntimeException: No lock daemon port
  java.lang.RuntimeException: No lock daemon port 
  at
com.ibm.ftt.rse.mvs.server.filesystem.impl.FFSServerImpl.sendCommandToLockDaemon
     (FFSServerImpl.java:325) 

Explanation

rsed.envvars is not updated to support the lock daemon. This situation applies to Rational Developer for System z Version 7.5.0.1 and higher only.

Update rsed.envvars to support the lock daemon.

Additional information

None.

EDC8128I Connection refused.

 

Message

ERROR FFSServerImpl: java.net.ConnectException: EDC8128I Connection refused. 
  java.net.ConnectException: EDC8128I Connection refused. 
  at java.net.PlainSocketImpl.socketConnect(Native Method) 

Explanation

This error message means that the lock daemon (FFSServerImpl) or JMON (JESMiner) is not listening on the designated port.

Additional information

See Language Environment Run-Time Messages (SA22-7566), message DC8128I.

Location: http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/CEEA9190/SPTCS00425 (z/OS 1.10)

IRRRacf (Not found in java.library.path)

 

Message

ERROR logHandler: java.lang.UnsatisfiedLinkError: IRRRacf (Not found in java.library.path)

Explanation

/usr/lib/libIRRRacf.so is not in LIBPATH, or /usr/lib/libIRRRacf.so is not accessible.

Additional information

This message is informational (INFO) in Developer for System z Version 7.5. The message is an error (ERROR) in Version 7.6.

pThreadSecurity failed

 

Message

ERROR class com.ibm.etools.zos.server.ZosSystemService:  
  java.lang.RuntimeException: pThreadSecurity failed, rc=7 
  at com.ibm.etools.zos.server.ZosSystemService.setThreadSecurity 

Explanation

If unsuccessful, pthread_security_np() sets errno to one the following values.

Note:
  • The case of /usr/lib/libIRRRacf.so not being program-controlled results in RC=7, case 1a in the following table.
  • The case of the client user not having read access to the FEKAPPL profile in the APPL class results in RC=6.

RC

Errno

Explanation

1

EACCES

The password or PassTicket, or the password phrase provided is not valid for the passed userid.

2

EINVAL

A parameter is not valid.

3

EMVSERR

An MVS environmental error or internal error occurred.

4

EMVSEXPIRE

The password or PassTicket, or the password phrase provided has expired.

5

EMVSSAF2ERR

The SAF call to the security product incurred an error.

6

EMVSSAFEXTRERR

The SAF call to the security product incurred an error.

7

EPERM

1a. The process does not have appropriate privileges to set a thread-level security environment.

Note: /usr/lib/libIRRRacf.so not being program controlled will result in case 1a.

1b. The caller is not permitted to the BPX.SERVER FACILITY class profile or BPX.SERVER is not defined and the caller is not a superuser.

1c. No password or PassTicket, or password phrase is provided and the caller is not defined as a surrogate of the passed user ID.

2. The caller is not a superuser and permitted to the BPX.DAEMON FACILITY class profile or BPX.DAEMON is not defined and the caller is not a superuser.

8

ESRCH

The user ID provided as input is not defined to the security product or does not have an OMVS segment defined.

10

ENOSYS

The function is not implemented.

100

?

new error, not mapped, see C/C++ Run-Time Library Reference (SA22-7821)

Additional information

See C/C++ Run-Time Library Reference (SA22-7821), section discussing pthread_security_np()

Location: http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/EDCLB190/3.740 (z/OS 1.10)

pthread_security_np() is a wrapper for z/OS UNIX service BPX1TLS/BPX4TLS. See http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/BPXZB1B0/2.142 (z/OS 1.12).

rsedaemon.log

[s]cmd=java [s]env

 

Message

cmd=java -D...

Explanation

When the RSE daemon starts, it lists the content of rsed.envvars in rsedaemon.log (line=''). This is followed by the thread pool startup information (cmd=java env[] ). Thread pools started dynamically to satisfy a connection request show up with s-markers (scmd=java senv[]).

Additional information

None.

rseserver.log

logHandler: pid:userid:ticket:port:clientid

 

Message

INFO logHandler: 66107:smile:4464456:8108-8118:9828744

Explanation

The message is logged at the start of RSE server initialization, before the client connects. The message is part of a sequence of messages produced during logon. Example:

INFO logHandler: 66107:smile:4464456:8108-8118:9828744  
  INFO logHandler: ProcessId=66107: port 8108 assigned to smile  
  INFO logHandler: ProcessId=66107: smile logged on 

Additional information

None.

logHandler: ProcessId=pid: port port assigned to userid

 

Message

INFO logHandler: ProcessId=66107: port 8108 assigned to smile

Explanation

The message is logged after the RSE server thread is initialized and the port number is sent back to the client. The message is not related to the client actually connecting.

Note: This message does not exist in Developer for System z Version 7.5.x.x. In that version, the message logHandler: ProcessId=pid: userid logged on is issued instead.

The message is part of a sequence of messages produced during logon. Example:

INFO logHandler: 66107:smile:4464456:8108-8118:9828744
INFO logHandler: ProcessId=66107: port 8108 assigned to smile
INFO logHandler: ProcessId=66107: smile logged on 

Additional information

None.

logHandler: ProcessId=pid: userid logged on

 

Message

INFO logHandler: ProcessId=66107: smile logged on

Explanation

Developer for System z Version 7.5: The message is logged after the RSE server thread is initialized and the port number is sent back to the client. The message is not related to the client actually connecting.

Developer for System z Version 7.6: The message is logged after the client actually connects.

Note: This message is enhanced with the user log location in Developer for System z Versions 7.5.1.4, 7.6.2.2, and 8.0.2. Example:
logHandler: ProcessId=pid: userid logged on 
with the user logs location of userlog.

The message is part of a sequence of messages produced during logon. Example:

INFO logHandler: 66107:smile:4464456:8108-8118:9828744
INFO logHandler: ProcessId=66107: port 8108 assigned to smile
INFO logHandler: ProcessId=66107: smile logged on

Additional information

None.

RseDaemon: startProcesses(): spawn command result=ppid:errno

 

Message

INFO RseDaemon: startProcesses(): spawn command result=156:132

Explanation

The message is logged when the daemon starts a thread pool.

Note:

Note: The parent-pid (ppid) is a child of the daemon, but it is also the parent of the actual thread pool process due to our double-kick startup mechanism.

The errno value picks up bogus values when the thread pool creation is successful. This problem is resolved in Developer for System z Version 8.0.2.0 (37988).

The layout of the message is changed in Version 8.0.2.0.

INFO RseDaemon: startProcesses(): spawn command: parentPID(672) errno(0) 
     dataFd(144) ctlFd(150)

The message is part of a sequence of messages produced during thread pool startup. Example:

INFO RseDaemon: startProcesses(): spawn command result=156:132
INFO RseDaemon: startProcesses(): send Query to the Server process
INFO RseDaemon: startProcesses(): Server process started. (processId=157)
INFO : ProcessId=157, Memory_Usage=1%, Used_Memory=987KB, threads=3, TCBs=12 

Additional information

None.

RseDaemon: startProcesses(): spawn command: parentPID(ppid) errno(errno) dataFd(fd) ctlFd(fd)

 

Message

INFO RseDaemon: startProcesses(): spawn command: parentPID(672)
     errno(0) dataFd(144) ctlFd(150) 

Explanation

The message is logged when the daemon starts a thread pool.

Note: The parent-pid (ppid) is a child of the daemon, but it is also the parent of the actual thread pool process due to our double-kick startup mechanism.

Layout of the message is different before Developer for System z Version 8.0.2.0.(37988). Example:

INFO RseDaemon: startProcesses(): spawn command result=ppid:errno

The message is part of a sequence of messages produced during thread pool startup. Example:

INFO RseDaemon: startProcesses(): spawn command: parentPID(156) 
     errno(0) dataFd(144) ctlFd(150)
INFO RseDaemon: startProcesses(): send Query to the Server process  
INFO RseDaemon: startProcesses(): Server process started. (processId=157)
INFO : ProcessId=157, Memory_Usage=1%, Used_Memory=987KB, threads=3, TCBs=12 

Additional information

None.

RseDaemon: getServerPort(): select timeout

 

Message

INFO RseDaemon: getServerPort(): select timeout

Explanation

The message is logged when the thread pool is unresponsive (normally during client logon) and is flagged with timeout status.

Note: The message comes at least 80 seconds after message:
INFO logHandler: pid:userid:ticket:port:clientid

The message is part of a sequence of messages produced during a client logon that is encountering problems. Example:

INFO logHandler: pid:userid:ticket:port:clientid 
INFO RseDaemon: getServerPort(): select timeout
INFO RseDaemon: selectThread poolProcess(): timeout, process=pid
INFO LogHandler: Thread poolMonitor:pid: command=TIMEOUT
INFO logHandler: ProcessId=pid: port port assigned to userid  

Additional information

None.

getServerPort(): select timeout

 

Message

INFO RseDaemon: getServerPort(): select timeout

Explanation

A server process to handle this new client connection was selected, but it does not respond in time. After three attempts, the process is deemed unusable and another or new server process is selected.

Additional information

None.

syslog

This section describes errors you might see in the syslog.

USER COMPLETION CODE=4083

 

Error

USER COMPLETION CODE=4083 REASON CODE=00000003

Explanation

The back chain was found in error.

See Language Environment Run-Time Messages (SA22-7566), section "Language Environment Abend Codes", code U4083 (X'FF3').

Location: http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/CEEA9190/8.13?SHELF=&DT=20080602034358 (z/OS 1.10)

Reason codes

03x - The save area chain terminated with a 0 chain pointer

Explanation: This abend and reason code, give during client connect or daemon IVP, usually indicates that APAR QO94976 for CA's ACF2 is not installed

Additional information: None.

ICH408I for USS based resources

  

Message

ICH408I USER(JOHN) GROUP(SYSPROG) NAME(JOHN DOE)

Explanation

Upon permission failure, RACF issues message ICH408I in the system log. RACF shows you what resource was requested on line 2, but with USS based resources, RACF does not tell you where exactly the permission failure occurred. The failure could be anywhere in the path leading to the requested resource. In the following example, /u/smile is allocated with 700, denying user JOHN access to /u/smile/.eclipse/RSE/SMILE.

ICH408I USER(JOHN) GROUP(SYSPROG) NAME(JOHN DOE)
/u/smile/.eclipse/RSE/SMILE 
CL(DIRSRCH) FID(01D5C9D4E5C8C200020B000000000003)
INSUFFICIENT AUTHORITY TO LSTAT
ACCESS INTENT(--X) ACCESS ALLOWED(GROUP ---)
EFFECTIVE UID(0096061081) EFFECTIVE GID(0999999999)

Use the looppath.sh shell script to list every directory of the path (looppath.sh <path> <cmd>).

$ /usr/lpp/rdz/bin/looppath.sh /u/smile/.eclipse/RSE/SMILE ls -dog
drwxr-xr-x 104 3904 Sep 9 11:41 /
dr-xr-xr-x 25 0 Oct 27 11:19 /u/ 
drwx------ 3 8192 Oct 2 2008 /u/smile/
drwxrwxrwx 3 8192 Jul 1 2008 /u/smile/.eclipse/
drwxrwxrwx 3 8192 Sep 3 2008 /u/smile/.eclipse/RSE/ 
drwxrwxrwx 2 8192 Jan 11 2009 /u/smile/.eclipse/RSE/SMILE/     

Additional information

None.

Client

RSEG1057

 

Message

RSEG1057 Connect failed. Host <host> not found or not responding

Explanation

Second logon with same user ID kills the first one (from Developer for System z Versions 7.5.1.1. and 7.6.0.1). See Multiple logons by a single userid.

Figure 2. Error message RSEG1057
Error message RSEG1057

Additional information

None.

RSEG1242

 

Message

RSEG1242 Daemon failed to launch server on <host> using port <port>.

Connection refused.

Explanation

"Connection refused" is a TCP/IP message saying that nobody is listening on the port you are trying to connect to. Here are the possible reasons no one is listening:

  1. RSE daemon is not active. Check the host. (For example, in SDSF.)
  2. RSE daemon is not using the port specified by the client. Check console message FEK002I (in the syslog or RSE daemon job output).
  3. A firewall or other network issue is blocking the client from accessing the RSE daemon port. (Port 0 in the RSEG1242 message normally indicates the RSE daemon.) Try telnet from the client machine to the RSE daemon port. (The connection will hang for 30 seconds if successful, but a firewall should display a message.)
    telnet <host> <port>
  4. The client can access the RSE daemon, but not the RSE server, due to firewall issues. (This situation is typically indicated by a non-zero port number in the RSEG1242 message.) Use _RSE_PORTRANGE in rsed.envvars to limit the ports being used, and open them in your firewall.

Figure 3. RSEG1242 with connection refused
RSEG1242 with connection refused

Message

RSEG1242 Daemon failed to launch server on <host> using port <port>

server failure: <message>

Explanation

The details show the error message shown in rseserver.log (only if the RSE daemon or server was reached). Host logs are probably needed to see what is going on and resolve the issue. See the rseserver.log section for more information.

Figure 4. RSEG1242 with server failure
RSEG1242 with server failure

Message

RSEG1242 Daemon failed to launch server on <host> using port <port>

userid: <message>

Explanation

The details show the error message shown in rseserver.log (only if RSE daemon or server was reached). Host logs are probably needed to see what is going on and resolve the issue. See rseserver.log section for more information.

Figure 5. RSEG1242 with user failure
RSEG1242 with user failure

Additional information

None.

RSEG1243

 

Message

RSEG1243 Failed to connect to the daemon on <host> using port <port> with an unexpected exception.

Explanation

This message indicates that there was an exception while launching the server using the daemon. The exception is shown in the detailed message.

java.net.SocketTimeoutException: Read timed out

The client was able to connect to RSE daemon at TCP/IP level, and it was able to send user ID or password, but the RSE daemon is not able to send something back (server port and ID token). This is probably a firewall issue, prohibiting outbound traffic. Or there are two client PCs on the network with the same IP address, and the answer is routed back to the other machine.

Figure 6. RSEG1243 error message
RSEG1243 error message

Additional information

None.

Security messages

R_ticketserv (IRRSPK) return codes

IRRPassTicket.generate

 

Message

ERROR class com.ibm.etools.zos.server.ZosClient: (R_ticketserv) 
  SafRc=8, racfRc=8 racfRsn=16
  at com.ibm.eserver.zos.racf.IRRPassTicket.generate 
ERROR class com.ibm.etools.zos.server.ZosSystemService:  
  SafRc=8, racfRc=8 racfRsn=16  at 
     com.ibm.eserver.zos.racf.IRRPassTicket.generate 

Explanation

When this error occurs during logon, it is due to a security setup error (or missing ACF2 or TSS maintenance). See PassTicket - Access not authorized (8/8/16) when it occurs randomly with RC 8/8/16 and the security product is ACF2 or TSS.

Additional information

See Security Server RACF Callable Services (SA22-7691), section R_ticketserv (IRRSPK00)

Location: http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/ICHZD190/2.48.7?SHELF=ICHZBK90&DT=20080520114546 (z/OS 1.10)

Table 2. R_ticketserv (IRRSPK00) return codes

R_ticketserv (IRRSPK00) return codes

SAF return code

RACF return code

RACF reason code

Explanation

0

0

0

The service was successful.

4

0

0

RACF is not installed.

8

8

0

Invalid function code.

8

8

4

Parameter list error.

8

8

8

An internal error was encountered.

8

8

12

A recovery environment could not be established.

8

8

16

Not authorized to use this service.

8

8

20

High order bit was not set to indicate last parameter.

8

12

8

Invocation of the Security Server Network Authentication Service Program Call (PC) interface failed with a 'parameter buffer overflow' return code. This indicates an internal error in IRRSPK00.

8

12

12

Invocation of the Security Server Network Authentication Service Program Call (PC) interface failed with an 'unable to allocate storage' return code. The region size for the Security Server Network Authentication Service started task (SKRBKDC) should be increased.

8

12

16

Invocation of the Security Server Network Authentication Service Program Call (PC) interface failed with a 'local services are not available' return code. This indicates that the Security Server Network Authentication Service started task (SKRBKDC) address space has not been started or is terminating.

8

12

20

Invocation of the Security Server Network Authentication Service Program Call (PC) interface failed with an 'abend in the PC service routine' return code. The symptom record associated with this abend can be found in the logrec data set.

8

12

24

Invocation of the Security Server Network Authentication Service Program Call (PC) interface failed with an 'unable to obtain control lock' return code. This can occur if the task holding the lock is not being dispatched (for example, a dump is in progress).

8

16

X'nnnn nnnn'

The Security Server Network Authentication Service was not able to successfully extract the client principal name from the supplied Kerberos V5 ticket. X'nnnnnnnn' is the Kerberos return code. Refer to the Security Server Network Authentication Service documentation for more information.

8

16

28

Unable to generate PassTicket.

8

16

32

PassTicket evaluation failure. Possible reasons include:

PassTicket to be evaluated is not a successful PassTicket.

The PassTicket to be evaluated was already evaluated before and replay protection is in effect.

No PTKTDATA profile exists to match the specified application.

An internal error occurred.

stderr.log

This section describes errors you might see in the stderr.log.

EDC5163I SAF/RACF extract error.

 

Message

pthread: EDC5163I SAF/RACF extract error.

Explanation

See Language Environment Run-Time Messages (SA22-7566), section discussing XL C/C++ Run-Time Messages, message EDC5163I.

Location: http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/CEEA9190/SPTCS00264 (z/OS 1.10)

An authorization failure occurred when attempting the service. This message is equivalent to the z/OS UNIX System Services errno, EMVSSAFEXTRERR.

This message is probably linked to the pThreadSecurity failed message in rsecomm.log.

Additional information

None.

EDC5111I Permission denied.

 

Message

pthread: EDC5111I Permission denied.

Explanation

See Language Environment Run-Time Messages (SA22-7566) , section discussing XL C/C++ Run-Time Messages, message EDC5111I.

Location: http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/CEEA9190/SPTCS00217 (z/OS 1.10)

An attempt was made to access a file in a way that violates its file access permissions. This message is equivalent to the POSIX.1 EACCES errno.

This message is probably linked to the pThreadSecurity failed message in rsecomm.log.

Additional information

None.

CA ACF2 / Top Secret (TSS)

TSS7250E equals ICH408I in RACF

 

Message

TSS7250E 151 J=<job/AS name> A=<userid> TYPE=<security class> RESOURCE=<resource to be checked>

Explanation

Unlike ICH408I, the message does not tell which security profile blocked the access, or which access was requested.

Additional information

None.

SAFOETKT U4083

 

Message

SAFOETKT U4083

Explanation

SAFOETKT U4083 abends in the syslog with client connects and tests with fekfivpd.

The following fixes are known to help with PassTicket problems when using ACF2/TSS.

Additional information

None.

Logon fails with RACF Internal Errors

 

Message

INFO LogHandler: ThreadSecurity error: rc=5

ERROR RseDaemon: getServerPort(): result=-1;RACF Internal Errors (FEKAPPL)

Explanation

  1. pthread_security_np() returns EMVSSAF2ERR (=5). ACF/2 detailed trace shows RC 8/8/4 on initACEE (Parameter list error occurred).

    The following fix resolves the ACF2 error:

  2. Due to an error in CA Top Secret, a _check_resource_auth_np() against the APPL class will fail with EMVSSAF2ERR if TSS defaults are used. As a result, the client fails with "RACF Internal Errors".

    The following fix resolves the TSS error:

Additional information

This problem occurs in Developer for System z Version 7.5.1.1, Version 7.5.1.2, Version 7.6.0.0, Version 7.6.0.1 and Version 7.6.1.0 only.

PassTicket - Access not authorized (8/8/16)

 

Message

syslog:

ACF04056 ACCESS TO RESOURCE IRRPTAUTH.FEKAPPL.userid TYPE 
         RPTK BY userid NOT AUTHORIZED
  

rseserver.log:

ERROR class com.ibm.etools.zos.server.ZosClient: (R_ticketserv)
SafRc=8, racfRc=8 racfRsn=16  at com.ibm.eserver.zos.racf.IRRPassTicket.generate

Explanation

Any command can cause this PassTicket generation error with ACF2 or Top Secret. The initial report was for JESminer but the problem may still exist for other Rational Developer for System z host code.

The JESMiner thread is running continuously until the client disconnects from the RSE Server. The JESMiner thread generates a PassTicket at the time of CONNECT request. When the same thread tries to generate a PassTicket again (for example, for reconnect), ACF2/TSS issues error message "Access not authorized" for the PassTicket function. This is because ACF2/TSS changes the effective ID to the client ID after the thread-level security is established for the client. On the other hand, RACF does not change the effective ID even if the thread-level security is changed for the client. So, RACF always generates the PassTicket under the authority of the started task ID, but ACF2/TopSecret tries to generate the PassTicket under the authority of the client ID after the thread-level security is established. The client ID does not have any authority to generate a PassTicket, and so SafRc=8, racfRc=8, racfRsn=16 (codes for "no authority") are issued.

The following Rational Developer for System z fix resolves the ACF2/TSS-specific problem:

In another occurrence, a change was made in Developer for System z Version 8.0.1 that resulted in two pthread_secutity_np() calls in a single thread. Due to the effective ID switching done by ACF2/TSS, the second call fails with return codes 8/8/16.

The following Developer for System z fix resolves this ACF2/TSS-specific problem:

Additional information

None.

64-bit PassTicket generation results in return codes 8/8/4

 

Message

ERROR ServerThread: 517: SafRc=8, racfRc=8 racfRsn=4
  com.ibm.eserver.zos.racf.IRRPassTicket.generate(IRRPassTicket.java:226)
ERROR RseDaemon: getServerPort(): result=0;server failure: SafRc=8, 
         racfRc=8 racfRsn=4 
  com.ibm.etools.zos.server.ZosSystemService.generatePassTicket
        (ZosSystemService.java:96) 
  com ibm.etools.zos.server.ServerThread.run(ServerThread.java:83) 
  java.lang.Thread.run(Thread.java:736)   

Explanation

Due to a bug in TSS and ACF2, Developer for System z cannot generate a PassTicket when 64-bit Java™ is used, and the call ends with SafRc=8, racfRc=8, racfRsn=4 (Invalid parameter list). The following PTFs resolve this issue:

Additional information

None.

TSS R14->R15 upgrade problem

 

Message

INFO LogHandler: ThreadSecurity error: rc=1 
ERROR RseDaemon: getServerPort(): result=-1;PassTicket Generation Error 

Explanation

A customer using 31-bit Rational Developer for System z Version 7.6.2.1 host got this error after upgrading TSS from Release 14 to Release 15.

The ThreadSecurity error: rc=1 indicates that our pthread_security_np() call failed with return code EACCES, which means "Permission is denied; the specified password is incorrect." This indicates things fail in the usage of the generated PassTicket, not the generation itself.

The SIGNMULTI flag (which is the TSS equivalent of the NO REPLAY PROTECTION used by RACF) was good:

TSS ADD(NDT) PSTKAPPL(FEKAPPL) SESSKEY(KEY16 ) SIGNMULTI 

CA TSS support indicated that a PTF will be supplied.

Additional information

None.

Caller not authorized to use getpsent

 

Message

ck_process_owner <userid> <groupid> <uid> <gid> 8 8 4
mm/dd/yy   yy.ddd h.mm.ss <jobname> <source> <sysname> <cpu>
Failed - Caller not authorized to use this callable service
Real Uid : 0 Effective UID: 0 
Saved UID : 0
Target PID : 0
Signal code : Signal n/a for type 3 (getpsent) requests
Signal code : 0

Explanation

The ISPF Client Gateway checks the active processes to see if there is already a server active from a previous invocation of the service. To do so, it analyses the output of the getpsent callable service. z/OS UNIX will try to collect information for every active process (including the ones the requester has no privileges for) and return only those for which the requestor is authorized.

So the error message (which is only visible in trace-like security reports) is related to how z/OS UNIX works. It is NOT hindering normal operation.

Additional information

For information, see http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/BPXZB190/2.248?SHELF=BPXZSH91&DT=20080521092140 (z/OS 1.10).

JMON joblog

JM227I Unexpected RACF error

 

Message

JM227I Unexpected RACF error. SAF 8, RACF 28, Reason 0, User GP1TAJ

Explanation

See Security Server RACROUTE Macro Reference (SA22-7692), section RACROUTE REQUEST=VERIFY

Location: http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/ICHZC690/3.61.2?SHELF=ICHZBK90&DT=20080603040614 (z/OS 1.10)

08 means that the requested function has failed.

28 means that the OIDCARD (operator ID) parameter is required but not supplied.

OIDCARD=oid addr specifies the address of the currently defined operator identification card of the user who has entered the system. The address points to a field that is 1 byte in length, followed by the operator ID card.

OIDCARD is used for operators logging on without password, and is a password substitution for them.

A possible scenario for getting this error is if JMON received a null password.

Additional information

None.

syslog

This section describes errors you might see in the syslog.

ABEND=SEC6

 

Message

ABEND=SEC6 U0000 REASON=0F01C008

Explanation

An error occurred during processing of a callable service to z/OS UNIX System Services.

See MVS System Codes (SA22-7626), section "System Completion Codes", code EC6.

Location: http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/IEA2H790/SPTM014775 (z/OS 1.10)

Reason code

Hints and tips

This chapter presents some useful hints and tips for setting up Rational Developer for System z.

Behavior changes

APPC no longer default

PassTicket setup required

Application security setup required

Multiple logons by a single userid

In Versions 7.5.1.1 and 7.6.0.1, there is a behavior change for multiple logons by a single userid. Until those release versions, Rational Developer for System z did not check for multiple logons by the same user, because only the user-specific log files are affected. However, some customers have users who regularly lose the client-host connection (usually because they are tired of waiting for a huge download and shut down the client). When those users log on again, the time-outs of the first session have not yet expired, and the host thinks the first session is still active. This is not a problem, except that the first session still has the lock on all the files the user was editing, and the users are not willing to wait for the host to realize that the session is gone and free the locks.

Lowering the time-out values was a first attempt to resolve the problem (in Version 7.6 GA), but this is a delicate balance, as Developer for System z should not terminate a session just because there is a network slowdown.

As a solution, the single.logon option was created (activated with v7511 UK51499 or v7601 UK54177) that cleans up any existing session if a user ID logs on twice. This is now the default behavior, because the benefit of ensuring all locks are freed outweighs the problem for the few users that log on multiple times. You can change this behavior with the new single.logon directive in rsed.envvars, which is documented in the PTF hold information.

The new directive looks like this:

#_RSE_JAVAOPTS="$_RSE_JAVAOPTS -Dsingle.logon=false"

The purpose is to allow a user ID to log on multiple times. The default value is false. Uncomment to allow a user ID to log on multiple times to a single RSE daemon.

Note: A second logon attempt will cause the first one to be cancelled by the host if this directive is not active or set to false. This cancellation is accompanied by console message FEK210I.

Log location

In Version 7.5.0.0, daemon.log was introduced in rsed.envvars (commented out) to specify where rsedaemon.log and rseserver.log are located. If not specified, the home directory of the user assigned to the RSED started task is used. Note that daemon.log is relative to the configuration directory (/etc/rdz) when it is not an absolute path.

In Version 7.5.1.1, user.log and DSTORE_LOG_DIRECTORY were introduced in rsed.envvars, (commented out) which specify where the user-specific logs are located: user.log/DSTORE_LOG_DIRECTORY/$LOGNAME/*.log.

In Version 7.6.0.0, Developer for System z uncommented the mentioned variables and routed all logs to /var/rdz/logs. Example:

_RSE_JAVAOPTS="$_RSE_JAVAOPTS -Ddaemon.log=/var/rdz/logs"
_RSE_JAVAOPTS="$_RSE_JAVAOPTS -Duser.log=/var/rdz/logs"
_RSE_JAVAOPTS="$_RSE_JAVAOPTS -DDSTORE_LOG_DIRECTORY=" 

Server bind address

Host based property groups

Host based projects

Debugging and tracing host issues

FTP to IBM

When you want to send files to IBM using FTP, use the following process:

  1. All files must be sequential. If the file is a PDS(E) or a single member, use XMIT to convert it to a sequential file. Example:
    XMIT DUMMY.DUMMY DS('data.set.name(member)') outdsn(userid.PMR#.XMIT')
  2. Bigger files must be tersed. Example:
    //TERSE    EXEC PGM=AMATERSE,PARM=PACK
    //*TERSE    EXEC PGM=TRSMAIN,PARM=PACK
    //*STEPLIB  DD DISP=SHR,DSN=SYS1.MIGLIB
    //SYSPRINT DD SYSOUT=*
    //INFILE   DD DISP=SHR,DSN=data.set.name
    //OUTFILE  DD DISP=(NEW,CATLG),SPACE=(CYL,(250,250),RLSE),
    //            UNIT=SYSALLDA,DSN=data.set.name.TRS
  3. Enter these commands from the TSO Ready prompt:
    1. ftp ftp.emea.ibm.com
    2. user ID: anonymous
    3. password: you@email.com (Use your e-mail address for the password.)
    4. cd toibm/mvs
    5. binary
    6. put 'userid.<name>.trs' pmr#.branch#.country#.<name>.trs
    7. quit

SVC dump

Because RSE is a z/OS UNIX process, the OMVS data is often needed to get the complete picture. RSE also contains multiple address spaces (daemon and thread pools), so wildcards are required.

manual

DUMP COMM=(description or dump name)
R xx,SDATA=(PSA,SQA,LSQA,RGN,TRT,LPA,CSA,SUM,ALLNUC),CONT
R xx,JOBNAME=(OMVS,RSED*),DSPNAME=('OMVS'.*,'RSED*'.*),END

slip trap

CD SET,SDUMP=(CSA,LPA,LSQA,NUC,PSA,RGN,SQA,SUM,SWA,TRT),Q=YES
CD SET,SDUMP,MAXSPACE=2000M
SLIP SET,ID=RSES,MSGID=FEKxxxI,JL=(OMVS,RSED*),DN=('OMVS'.*,'RSED*'.*),END  

APPC TP data

The APPC white paper, Using APPC to provide TSO command services (SC14-7291-00), holds more information on setting up APPC itself, setting up the APPC transaction program (TP), modifying an existing TP, and basic problem solving.

APPC TP JES messages

When a TP executes, the TP runtime messages, such as allocation and termination messages, go to a log named by the MESSAGE_DATA_SET keyword in its TP profile. The default value in sample JCL FEK.SFEKSAMP(FEKAPPCC) is &SYSUID.FEKFRSRV.&TPDATE.&TPTIME.LOG. This log file does not appear unless the KEEP_MESSAGE_LOG(ALWAYS) keyword is added to the transaction definitions. The default value in sample JCL FEK.SFEKSAMP(FEKAPPCC) is ERROR.

APPC TP SYSTSPRT

For actual debugging (only on request of IBM support), you can trap the SYSTSPRT output of the transaction. The default output location in sample JCL FEK.SFEKSAMP(FEKAPPCC) is SYSOUT=*.

When writing to a dataset, be sure you use a user-specific naming convention (for example, use &SYSUID. in the DSN), because there will be one APPC transaction per Rational Developer for System z user.

When writing the output to SYSOUT, be aware that JES treats APPC's SYSOUT output as special:

APPC trace

APPC API TRACE (text file)

To perform an APPC API trace to a text file, use the following process:

  1. Allocate a sequential dataset with VB 27994 27998, CYL(1,1), for example userid.APITRACE.
  2. Ensure APPC has ALTER access to userid.APITRACE.
  3. From ISPF option 6 or TSO READY mode, issue the command:
    EX 'SYS1.SBLSCLI0(ATBTRACE)' 'START DSN(userid.APITRACE) 
        LU(base_lu) TP(FEKFRSRV)'
  4. Reproduce the failure
  5. From ISPF option 6 or TSO READY mode, issue the command:
    EX 'SYS1.SBLSCLI0(ATBTRACE)' 'STOP DSN(userid.APITRACE)'

The trace is case-sensitive.

APPC CTRACE (SVCDUMP)

To perform an APPC CTRACE(SVCDUMP), use the following process:

  1. Issue command: TRACE CT,256M,COMP=SYSAPPC
  2. Issue command: R xx,OPTIONS=(GLOBAL),END
  3. Reproduce the failure.
  4. Issue command: TRACE CT,OFF,COMP=SYSAPPC

ISPF ISPVCALL trace

Add isptrace=nullfile to ISPF.conf. This will tell ISPF to start tracing. Output is written to userid.ISPVCALL.TRACE ( or to userid.CALL.TRACE if the z/OS release level is before 1.9).

To test if ISPVCALL works as expected, issue: TSO ISPVCALL STATUS. (No DD is needed.)

TCP/IP RESOLVER trace

Location of documentation for this trace: http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/F1A1C590/3.33.2?SHELF=F1A1BKB1&DT=20090402133646 (z/OS 1.11).

Add the following variable setting to rsed.envvars:

RESOLVER_TRACE=/var/rdz/logs/resolver.trace 

You can use RESOLVER_TRACE=stdout when you are working from the command line.

TCP/IP and RESOLVER CTRACE

To perform a TCP/IP and RESOLVER CTRACE, use the following process:

  1. Start external writer:
    TRACE CT,WTRSTART=CTWTR1
  2. Start resolver CTRACE:
     TRACE CT,ON,COMP=SYSTCPRE,SUB=(resolverprocname) 
       R xx,WTR=CTWTR1,OPTIONS=(ALL),END
  3. Start TCPIP CTRACE:
    TRACE CT,ON,COMP=SYSTCPIP,SUB=(tcpipprocname)   
      R xx,WTR=CTWTR1,OPTIONS=(TCP,PFS,UDP),END
  4. Recreate the problem.
  5. Stop the traces:
    TRACE CT,ON,COMP=SYSTCPRE,SUB=(resolverprocname)   
      R xx,WTR=DISCONNECT,END  
    TRACE CT,OFF,COMP=SYSTCPRE,SUB=(resolverprocname)  
    TRACE CT,ON,COMP=SYSTCPIP,SUB=(tcpipprocname)   
      R xx,WTR=DISCONNECT,END  
    TRACE CT,OFF,COMP=SYSTCPIP,SUB=(tcpipprocname)  
    TRACE CT,WTRSTOP=CTWTR1,FLUSH

SAF trace

The SAF trace documented here requires a diagnostic driver for Developer for System z that creates a console message just before and after the targeted security call, in this case PassTicket generation.

The diagnostic driver issues the following messages when a PassTicket is generated:

Follow these instructions to activate the test driver outside SMP/E.

  1. Make a backup of /usr/lpp/rdz/lib/zosserver.jar:
    mv /usr/lpp/rdz/lib/zosserver.jar /usr/lpp/rdz/lib/zosserver.jar.bu
  2. FTP the attached zosserver.jar in binary to /usr/lpp/rdz/lib/ (59779 bytes).
  3. Make the uploaded file executable:
    chmod 755 /usr/lpp/rdz/lib/zosserver.jar
  4. Verify the change has been made:
    ls -lE /usr/lpp/rdz/lib/zosserver.jar
    -rwxr-xr-x --s- 1 IBMUSER SYS1 59779 Dec 17 12:04 zosserver.jar
  5. Restart the RSED started task to pick up the changes.

The next thing to do is set up the system dump and SAF trace. Note that RACF relies on GTF to create a SAF trace.

See (for z/OS 1.10)http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/IEA2V191/10.0?SHELF=IEA2BK91&DT=20090220211519

  1. Create a GTF started task, or use an existing one:
    //*
    //* GTF TRACE
    //*
    //* START COMMAND : S GTF.GTFUSER
    //* STOP COMMAND : P GTFUSER
    //*
    //GTF PROC TRACE=USER.$$$$$GTF.TRACE,
    // DSP=NEW,
    // SIZE='CYL,10',
    // PLIB=USER.$$$$$GTF.PARMLIB,
    // MEMBER=GTFPARM
    //*
    //IEFPROC EXEC PGM=AHLGTF,TIME=1440,REGION=4096K,
    // PARM='MODE=EXT,DEBUG=NO,TIME=YES'
    //IEFRDER DD DSN=&TRACE,
    // DISP=(&DSP,CATLG),
    // UNIT=SYSALLDA,
    // SPACE=(&SIZE)
    //SYSLIB DD DISP=SHR,DSN=&PLIB(&MEMBER)
    //*
  2. Create a GTF trace options file, for example GTFSAF, with the following content:
    TRACE=USRP
    USR=(F44)
    END
  3. Create a slip trap that triggers on the FEKP02I message (console command):
     SLIP SET,ID=RDZ1,MSGID=FEKP02I,JL=(RACF,RSED*),DN=(1.IRR*),END
  4. Start the GTF trace (console command):
    START GTF.GTFRDZ,MEMBER=GTFSAF
  5. Accept the trace options by replying to AHL125A RESPECIFY TRACE OPTIONS OR REPLY U (console command):
    R xx,U
  6. Enable the SAF trace in RACF (console command):
    @SET TRACE( JOBNAME(RSED*) RACROUTE(TYPE(1)) CALLABLE(TYPE(43)) ) LIST
    Note: The "@" represents the subsystem command prefix you assigned to RACF via the INITPARM value in SYS1.PARMLIB(IEFSSNxx). This implies that the RACF subsystem must be active.

    For more information about this, see http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/ICHZA290/4.6.1.1?SHELF=ICHZBK90&DT=20080521121606&CASE= (z/OS 1.10). RACF support will tell you the desired trace options for the specific problem.

Recreate the problem. (Only logon, do not disconnect to avoid trace pollution.)

Finally, disable the trace, collect the data, and undo the changes for test driver.

Disable the trace as soon as possible after the problem has been recreated, to prevent the data from "rolling out" of the trace file.

  1. Disable the SAF trace in RACF (console command):
    @SET TRACE( NOJOBNAME NORACROUTE NOCALLABLE ) LIST
  2. Stop the GTF trace (console command):
    STOP GTFRDZ
  3. Terse and FTP the dump and trace to IBM. (Be sure to reference the PMR number.)
  4. Remove the test driver:
    mv /usr/lpp/rdz/lib/zosserver.jar.bu /usr/lpp/rdz/lib/zosserver.jar
  5. Verify the work of the previous command has been done:
     ls -lE /usr/lpp/rdz/lib/zosserver.jar
    -rwxr-xr-x --s- 2 IBMUSER SYS1 59669 Dec 16 18:37 zosserver.jar
  6. Restart the RSED started task to pick up the changes.

Show active z/OS UNIX processes

The SDSF PS view is an easy way of seeing z/OS UNIX processes (active and zombie), but there are other options.

A super user (UID 0) can run ps -ef from the OMVS shell to see all active processes, or ps -fu <userid> for those processes belonging to a specific user. For more information, see http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/BPXZA5B0/PS) (z/OS 1.12).

Operator commands D OMVS,A=ALL and D OMVS,U=<userid> give similar output as the ps shell command. See http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/IEA2G1B1/4.10.48 (z/OS 1.12).

Show z/OS UNIX reason code

To show the z/OS UNIX reason code, add the following variable setting to rsed.envvars:

_EDC_ADD_ERRNO2=1 

For more information, see the XL C/C++ Programming Guide (SC09-4765), section about _EDC_ADD_ERRNO2. (Location: http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/CBCPG190/4.12.2.7?SHELF=CBCBS190&DT=20080616063107 for z/OS 1.10)

When _EDC_ADD_ERRNO2 is set to 1, errno2 is added to perror(), strerror(), and strerror_r() messages. Example:

+FEK117E socketpair() failed. 
   reason=(EDC5124I Too many open files. (errno2=0x050B0146)) 

Note: _EDC_ADD_ERRNO2=1 is active by default in rsed.envvars in Developer for System z Version 8.0.1 and later.

GSK SSL trace

GSK SSL (also known as System SSL) is used by the RSE daemon. The RSE server uses Java SSL.

Set the following variables in rsed.envvars:

GSK_TRACE_FILE=/var/rdz/logs/gskssl.trc
GSK_TRACE=0xFF
Note: By default, the GSK_TRACE is written to /tmp/gskssl.%.trc, where % is the process ID.

Recreate the problem.

Format the trace. (Trace is sent to DD STDOUT.)

//GSKTRACE JOB CLASS=A,MSGCLASS=A,MSGLEVEL=(1,1),NOTIFY=&SYSUID
//*
//* SHOW FORMATTED GSKTRACE
//*
// SET FILE='/var/rdz/logs/gskssl.trc'
//*
//FORMAT EXEC PGM=BPXBATCH,REGION=0M,TIME=NOLIMIT,
// PARM='SH ls -l &FILE;echo;gsktrace &FILE'
//STDOUT DD SYSOUT=*
//STDERR DD SYSOUT=*
//*

Expected output:

SSL Alert messages

Alert messages, as in the following sample, are documented in RFC 2246 (http://www.ietf.org/rfc/rfc2246.txt).

ERROR read_v3_alert(): SSL V3 alert 46 received from <ip> ERROR
gsk_secure_socket_init(): SSL V3 server handshake failed with ...INFO 
default_setsocketoptions(): TCP_NODELAY restored for socket ...EXIT 
gsk_secure_socket_init(): <--- Exit status 0x0000019e (414)
       ENTRY gsk_strerror(): 
--->  Thd-0 EXIT gsk_strerror(): <--- Exit status 0x00000000 (0)

Alert

Description

Explanation

0

close_notify

This message notifies the recipient that the sender will not send any more messages on this connection. The session cannot resume if any connection is terminated without proper close_notify messages with level equal to warning.

10

unexpected_message

An inappropriate message was received. This alert is always indicates an unrecoverable error and should never be observed in communication between proper implementations.

20

bad_record_mac

This alert is returned if a record is received with an incorrect MAC. This message always indicates an unrecoverable error.

21

decryption_failed

A TLSCiphertext decrypted in an invalid way. Either it was not an even multiple of the block length or its padding values, when checked, were not correct. This message always indicates an unrecoverable error.

22

record_overflow

A TLSCiphertext record was received which had a length more than 2^14+2048 bytes, or a record decrypted to a TLSCompressed record with more than 2^14+1024 bytes. This message always indicates an unrecoverable error.

30

decompression_failure

The decompression function received improper input (for example, data that would expand to excessive length). This message always indicates an unrecoverable error.

40

handshake_failure

Receiving of a handshake_failure alert message indicates that the sender was unable to negotiate an acceptable set of security parameters with the available options. This is an unrecoverable error.

41

? no_certificate ?

 See SSL support.

42

bad_certificate

A certificate was corrupt, contained signatures that did not verify correctly, or had other problems.

43

unsupported_certificate

A certificate was of an unsupported type.

44

certificate_revoked

A certificate was revoked by its signer.

45

certificate_expired

A certificate has expired or is not currently valid.

46

certificate_unknown

Some other (unspecified) issue arose in processing the certificate, rendering it unacceptable.

47

illegal_parameter

A field in the handshake was out of range or inconsistent with other fields. This always indicates an unrecoverable error.

48

unknown_ca

A valid certificate chain or partial chain was received, but the certificate was not accepted because the CA certificate could not be located or could not be matched with a known, trusted CA. This message always indicates an unrecoverable error.

49

access_denied

A valid certificate was received, but when access control was applied, the sender decided not to proceed with negotiation. This message always indicates an unrecoverable error.

50

decode_error

A message could not be decoded because some field was out of the specified range or the length of the message was incorrect. This message always indicates an unrecoverable error.

51

decrypt_error

A handshake cryptographic operation failed, including being unable to correctly verify a signature, decrypt a key exchange, or validate a finished message.

60

export_restriction

A negotiation not in compliance with export restrictions was detected. For example, attempting to transfer a 1024 bit ephemeral RSA key for the RSA_EXPORT handshake method. This message always indicates an unrecoverable error.

70

protocol_version

The protocol version the client has attempted to negotiate is recognized, but not supported. (For example, old protocol versions might be avoided for security reasons.) This message always indicates an unrecoverable error.

71

insufficient_security

Returned instead of handshake_failure when a negotiation has failed specifically because the server requires ciphers more secure than those supported by the client. This message always indicates an unrecoverable error.

80

internal_error

An internal error unrelated to the peer or the correctness of the protocol makes it impossible to continue (such as a memory allocation failure). This message always indicates an unrecoverable error.

90

user_canceled

This handshake is being canceled for some reason unrelated to a protocol failure. If the user cancels an operation after the handshake is complete, just closing the connection by sending a close_notify is more appropriate. This alert should be followed by a close_notify. This message is generally a warning.

100

no_renegotiation

Sent by the client in response to a hello request or by the server in response to a client hello after initial handshaking. Either of these would normally lead to renegotiation. When renegotiation is not appropriate, the recipient should respond with this alert. At that point, the original requester can decide whether to proceed with the connection. One case where this would be appropriate would be where a server has spawned a process to satisfy a request; the process might receive security parameters (key length, authentication, and so forth) at startup, and it might be difficult to communicate changes to these parameters after that point. This message is always a warning.

Java SSL trace

Java SSL is used by the RSE server. The RSE daemon uses GSK SSL (also known as System SSL).

Set the following variables in rsed.envvars:

_RSE_JAVAOPTS="$_RSE_JAVAOPTS -Djavax.net.debug=true"
_RSE_JAVAOPTS="$_RSE_JAVAOPTS -Denable.standard.log=true"

The output is written to stdout, so enable.standard.log must be set to trap it. The trapped output will go to /var/rdz/logs/stdout.n.log ( where n=1, 2, 3, ...).

Java method trace

You can use the method trace to trace the NIO (network I/O) code within the Java Virtual Machine (JVM). Add the following parameters to the JVM:

This will produce a binary file nio.trc, which needs to be formatted using the following command:

java com.ibm.jvm.format.TraceFormat 
   /var/rdz/logs/nio.trc /var/rdz/logs/nio_trace.txt

The tracing suggested will trace only the SocketChannel class. If you want to trace the entire network I/O code, you need to limit the tracing to java/nio/* in the Xtrace command.

Note: Xtrace greatly reduces the performance.

CARMA memory leak trace

This trace requires Developer for System z Versions 7.1.1.4, 7.5.1.1, 7.6.0.1 or later.

CARMA can run with an LE diagnosis heap manager, which will allow development to find the cause of a memory leak. Upon termination, this heap manger writes a memory allocation report to the CARMALOG DD. See http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/CEEA1190/2.1.10?SHELF=CEE2BK90&DT=20080530025919 (z/OS 1.10) for more information.

Batch submit invocation CRASUBMT:

//CARMALOG DD SYSOUT=*
//SYSTSIN DD *  ISPSTART PGM(CRASERV) +  
  PARM('ENVAR("_CEE_HEAP_MANAGER=CEL4MCHK")/&PORT &TIMEOUT')
/* 

CRASTART invocation in crastart.conf (indentation indicates line continuation):

-COMMAND=ALLOC FI(CARMALOG) MOD CATALOG DSORG(PS) RECFM(F,B)
   LRECL(133) BLKSIZE(27930) SPACE(5,5) TRACKS UNIT(SYSALLDA)
   DA(&CRAUSER..&SYSNAME..CRA.CARMALOG)
PROGRAM=IKJEFT01 ISPSTART PGM(CRASERV)
   PARM('ENVAR("_CEE_HEAP_MANAGER=CEL4MCHK")/&CRAPRM1. &CRAPRM2.')             

Installation and setup

For information about how to set up and configure Rational Developer for System z, see the IBM Rational Developer for System z Host Configuration Guide (SC23-7658).

FEKFIVPT invocation results in initialization failure

 

Problem

Running the RSE Daemon with IVP, the FEKFIVPT gets the following error in the syslog:

Error

TCP/IP IVP test... 
Initialize failed, rc= 1004

Explanation and resolution

This error code is generated by the following invocation in FEKFIVPT:

parse value socket('Initialize', 'RSCLIENT') with rc .  
 if rc <>0 then call SayError 'Initialize' rc 0

The rc=1004 for a SOCKET API is documented in the TCPIP errno.h file, and it means:

1004 EIBMIUCVERR All The request failed because of an IUCV error.

This error is generated by the client stub code. Ensure IUCV/VMCF is functional.

Additional information

Here is a link to the z/OS Communications Server Version 1.9 that contains this socket return code information: http://publib.boulder.ibm.com/infocenter/zos/v1r9/index.jsp?topic=/com.ibm.zos.r9.cs3cod0/syserret.htm

You should ensure that the TCP/IP IUCV/VMCF subsystem is configured and started according to the TCP/IP stack that is used. Here is a link to the z/OS Communications Server Version 1.9 Information Center. The information there explains how to configure and start this subsystem: http://publib.boulder.ibm.com/infocenter/zos/v1r9/index.jsp?topic=/com.ibm.zos.r9.halz002/f1a1b37042.htm. Here is a link to the IUCV/VMCF considerations information: http://publib.boulder.ibm.com/infocenter/zos/v1r9/index.jsp?topic=/com.ibm.zos.r9.halz002/f1a1b37042.htm

IVP test results in "CEE3201S operation exception" error

 

Problem

Attempts to run an IVP test using IBM Rational Developer for System z using the command fekfivpd port username results in the error message:

CEE3201S The system detected an operation exception (System Completion Code=OC1)

Error

The full error message is:

CEE3201S The system detected an operation exception 
  (System Completion Code=OC1).
From entry point main at compile unit offset -33C7C3B0 at entry offset 
  -33C7C3B0 at address 00000000.
Possible Bad Branch: Statement: Offset: +000000E8
41 *-* 'set -a;.' path'rsed.envvars;fekfdivp' 
  port NoDollar(uid)  +++ RC(132) +++ 

Explanation and resolution

The TSO region size is too small.

Additional information

None.

Starting the RSE daemon task fails with error message BPXM009I

 

Problem

Attempts to start the RSE daemon task on the host results in the following message in the job output:

Error

BPXM009I BPXBATCH FAILED BECAUSE OPEN (BPX1OPN) FOR STDIN FAILED
   WITH RETURN CODE 0000009C REASON CODE 0B0C00FC

Explanation and resolution

Look at the last 2 bytes of the REASON CODE, which are 00FC. See "Reason Codes" in UNIX System Services Messages and Codes.

JRSAFNoUID: The user ID has no UID.

Create an OMVS segment with a UID. The problem is the user ID for the RSE started task has no group ID, and the OMVS segment is not defined correctly.

Additional information

None.

FEKFIVPI failing with MSG: ISPZINT not found

 

Problem

After installing Rational Developer for System z Version 7.5 and running the Installation Verification Program (IVP) the following output is generated:

Error

*** CHECK : USS MODULES  Checking ISPF Directory :
/usr/lpp/ispf  Checking modules in /usr/lpp/ispf/bin directory  RC=8  
MSG: ISPZINT not found
MSG: Remove suffix /bin from _CMDSERV_BASE_HOME setting in rsed.envv
Checking for ISPF configuration file ISPF.conf 
*** ERRORS FOUND IN INSTALLATION VERIFICATION CHECK ***
IVP processing terminated until errors corrected 
Re-run IVP process once errors above rectified
Host installation verification terminated with Errors 

Explanation and resolution

The ISPF SISPLPA data set could not be found or was not APF-authorized.

The SISPLPA data set must be:

Additional information

None.

fekfivpd when RSE started task is not active

When the RSE started task is not active, you might see the following messages:

connect() failed: EDC8128I Connection refused.
43 *-* 'set -a;.' path'rsed.envvars;fekfdivp' port NoDollar(uid) 
  +++ RC(255) +++  

fekfivpd with insufficient memory

You might see the following error messages in an "insufficient memory" situation:

CEE3201S The system detected an operation exception (System Completion Code=0C1)
From entry point main at compile unit offset -33C7C3B0 at entry offset -33C7C3B0
   at address 00000000.
Possible Bad Branch: Statement: Offset: +000000E8
<> LEAID ENTERED (LEVEL 10/15/2007 AT 09.21)
<> LEAID PROCESSING COMPLETE. RC=0 
41 *-* 'set -a;.' path'rsed.envvars;fekfdivp' port NoDollar(uid) 
+++ RC(132) +++ 

Generating pass ticket results in ERROR ServerThread: 517: SafRc=8

 

Problem

CA TSS or ACF is unable to generate a pass ticket using 64-bit Java.

Error

The full error message is:

ERROR ServerThread: 517: SafRc=8, racfRc=8 racfRsn=4 
ERROR RseDaemon: getServerPort(): result=0;server failure: 
      SafRc=8, racfRc=8 racfRsn=4
com.ibm.eserver.zos.racf.IRRPassTicket.generate(IRRPassTicket.java:226)
com.ibm.etools.zos.server.ZosSystemService.generatePassTicket
     (ZosSystemService.java:96) 
com.ibm.etools.zos.server.ServerThread.run(ServerThread.java:83)
java.lang.Thread.run(Thread.java:736)  

Explanation and resolution

Maintenance for Computer Associates Top secret or ACF2 is needed.

View the file rseserver.log to see if the above messages exist:

CA TSS R14.0 use PTF RO31780
CA TSS R15.0 use PTF RO30836 
CA ACF2 R14.0 use PTF RO31548
CA ACF2 R15.0 use PTF RO30898

Additional information

None.

RSED start with IVP parameter fails with Access Error

 

Problem

Similar error messages occur in various tests. RSED start with IVP parameter is failing with access error to IRRPTAUTH.FEKAPPL.* profile in the PTKTDATA security class.

Error

JES Job Monitor test:

executed on SYSTEM -- Mon May 16 14:46:35 EDT 2011
executed by uid=130154(STCRSE) gid=839(STARTASK)
using /etc/rdz/rsed.envvars 
current address space size limit is 1084203008 (1034.0 MB)
maximum address space size limit is 1084203008 (1034.0 MB)
testing JES Job Monitor on port 6715...
Initialize failed, rc= 1004 

Lock Daemon test:

executed on SYSTEM -- Mon May 16 14:46:36 EDT 2011 
executed by uid=130154(STCRSE) gid=839(STARTASK)
using /etc/rdz/rsed.envvars 
current address space size limit is 1084203008 (1034.0 MB)
maximum address space size limit is 1084203008 (1034.0 MB)
testing RSE Lock Daemon on port 4036...
Initialize failed, rc= 1004 

TCP/IP IVP test:

Initialize failed, rc= 1004 

PassTicket IVP test:

the default applid=FEKAPPL 
Access Error to IRRPTAUTH.FEKAPPL.* profile in PTKTDATA security class:
   SAF RC=4 ; RACF RC=4 ; RACF Reason=0
 RSE daemon IVP ended -- return code 40 --

Explanation and resolution

The RC=1004 is caused because the TCPIP stack could not be found.

Access Error to IRRPTAUTH.FEKAPPL.* profile in PTKTDATA is caused because PTKTDATA was not in the GENERIC PROFILE CLASS.

RC=1004 can be resolved by changing the socket initialization in FEKFIVPJ.

Parse value socket('Initialize', 'RSCLIENT') with return code. Parse value socket('Initialize', 'RSCLIENT',40,TCPIP stack name) with return code.

To make the client connection work, _BPXK_SETIBMOPT_TRANSPORT must be set in rsed.envvars pointing to the correct TCPIP stack.

The Access error can be resolved with: SETR GENERIC(PTKTDATA), followed by SETR RACLIST(PTKTDATA) REF.

Additional information

None.

Host node name cannot be found

 

Problem

During execution of the fekfivpt IVP or when starting the RSE daemon, using parameter IVP=IVP, you might see the wrong name for this host system.

host IP address:
 ------------------
hostName=NODENAME

Error

That NODENAME is wrong. The NODENAME should be the name of the user’s mainframe. The result should look similar to the example in the Host Configuration Guide (SC23-7658), "Installation verification" chapter, section "Verify started tasks", subsection "RSED, RSE daemon", in the STDOUT example, near the end of the example. (In the example, CDFMVS08 appears in place of NODENAME.).

Explanation and resolution

First, type this command at a USS prompt: hostname -r

If that gives the wrong value, then Rational Developer for system z will not work either. You might (wrongly) see:

EZZ8342I gethostbyname(NODENAME): Unknown host 

Second, read Appendix B of the Host Configuration Reference Guide (SC14-7290). That tells how to set up TCP/IP and explains how the searching works.

If nothing in Appendix B helps, then view your SYS1.PARMLIB(COMMNDxx) for this line:

S EZAZSSI,P=NODENAME 

If that line is present, then change NODENAME to the correct one. You might use the symbolic value &SYSNAME.:

S EZAZSSI,P='&SYSNAME.' 

Additional information

None.

Developer for System z client fails to connect to host using SSL

 

Problem

When using SSL with IBM Rational Developer for System z, the client connection fails. The security configuration of this customer included an unknown Certificate Authority (an internal created CA, not Thawte or Verisign). This is similar to using a self-signed certificate and requires that the public portion of the CA certificate must be extracted and added to the keyring as well as the requested certificate.

Error

A stack trace will show:

java.net.SocketTimeoutException: Read timed out
* 

In rseserver.log, this error will be seen:

ERROR CertificateValidator: 
java.io.FileNotFoundException: RSEDRING (EDC5129I No such file or 
  directory. (errno2=0x05620062))
  at java.io.FileInputStream.<init>(FileInputStream.java:123)
  at java.io.FileInputStream.<init>(FileInputStream.java:83) 
  at com.ibm.etools.zos.server.ZosCertificateValidator.<init>
   (ZosCertificateValidator.java:174)
  at com.ibm.etools.zos.server.RseDaemon.main(RseDaemon.java:358)  

and

INFO LogHandler:
ServerKeyStore:java.io.IOException: 
  Failed validating certificate paths  

Explanation and resolution

The RSE Server performs a validation of the certificate. In this case validation was failing. Add the internal issuing Certificate Authority certificate into the KEYRING used for RSED as well as the requested certificate.

Additional information

None.

RSEG1242 "Daemon failed to launch server on <host> using port <port>"

 

Problem

In IBM Rational Developer for System z, the message

RSEG1242 Daemon failed to launch server on <host> using port <port>

can have the following details:

Error

When you see CONNECTION REFUSED, then RSE (Remote System Explorer) daemon is not active.

When you see SERVER FAILURE, with a message, then your client has contacted the host but became disconnected.

Explanation and resolution

For CONNECTION REFUSED you should do the following things:

  1. Check the RSED job output in SDSF (Spool Display and Search Facility). The RSE daemon is not using the port specified by the client, or RSE daemon is not running.
  2. Check console message FEK002I (system log or RSE daemon job output). A firewall or other network issue is blocking the client from accessing the RSE daemon port. (Port 0 in the RSEG1242 message normally indicates the RSE daemon.)
  3. Try telnet from the client machine to the RSE daemon port (the connection will hang for 30 seconds if successful, but a firewall should display a message).
    telnet <host> <port>
    The client can access RSE daemon, but not the RSE server due to firewall issues. (This is typically indicated by a non-zero port number in the RSEG1242 message.)
  4. Alter the _RSE_PORTRANGE in /etc/rdz/rsed.envvars.

For SERVER FAILURE: <message>: The details show the error message shown in rseserver.log, but only if the RSE daemon or server was reached:

ERROR RseDaemon: server failure: password revoked 

  1. In this case, view detailed host logs to see which part of the code failed.
  2. Set debug_level=2 in /etc/rdz/rsecomm.properties and restart the RSE daemon, or use the operator commands to dynamically activate tracing:
    F RSED,APPL=RSL ON
    and
     F RSED,APPL=RCL ON
  3. After recreating the problem, customize and run FEK.SFEKSAMP(FEKLOGS). Create a service request PMR and send the output to IBM.

Additional information

None.

Access Error to IRRPTAUTH.FEKAPPL.* profile in PTKTDATA security class

 

Problem

The RSED started task running with the IVP=IVP argument failed.

Error

Access Error to IRRPTAUTH.FEKAPPL.* profile in PTKTDATA 
  security class : SAF RC=8 ; RACF RC=8 ; RACF Reason=0 

Explanation and resolution

Security setup is not complete. With this customer, the started task user ID OMVS segment did not have a GID assigned, and SETROPTS LIST showed

ATTRIBUTES = INITSTATS NOWHEN(PROGRAM) SAUDIT CMDVIOL NOOPERAUDIT 

with NOWHEN(PROGRAM) program control is not active. z/OS UNIX requires program control for using privileged functions.

Generate GID for Rational Developer for System z users and activate basic program control support. Both actions are documented in the Security setup documentation of Developer for System z :

ATTRIBUTES = INITSTATS WHEN(PROGRAM -- BASIC) SAUDIT CMDVIOL
NOOPERAUDIT 

Error

Access Error to IRRPTAUTH.FEKAPPL.* profile in PTKTDATA 
  security class : SAF RC=4 ; RACF RC=4 ; RACF Reason=0 

Explanation and resolution

The access error to the IRRPTAUTH.FEKAPPL.* profile in PTKTDATA is caused because PTKTDATA was not in the GENERIC PROFILE CLASS.

The Access error is resolved by doing the following things:

  1. Deleting the profile:
    RDELETE (PTKTDATA IRRPTAUTH.FEKAPPL.*)
  2. Making the class generic:
    SETROPTS GENERIC(PTKTDATA) 
  3. Redefining the profile:
    RDEFINE PTKTDATA IRRPTAUTH.FEKAPPL.* UACC(NONE
  4. Activating the changes:
    SETROPTS RACLIST(PTKTDATA) REFRESH

Additional information

None.

Installation and setup for CARMA and Endevor®

This section discusses some of the problems that can occur during CARMA installation and setup.

Return code -3 when starting the CARMA server

Problem

You get a return code -3 when starting the CARMA server.

Error

Look in the CARMA server job output in SDSF - cra&port# and see:

--- CRANDVRA CMD START 16:46:22 ----------------- 
invoking NDVRC1 PGM(CRASERV) PARM(13935 420) 
136 *-* Cmd 
+++ RC(-3) +++ 
--- CRANDVRA CMD END 16:46:22 ----------------- 

Explanation and resolution

Usually, the invocation fails with RC -3 when there is an APF authorization issue. Endevor must be APF-authorized, and when it is accessed using STEPLIB or TASKLIB, all other libraries in STEPLIB or TASKLIB, such as data set hlq.SFEKLOAD, must also be APF-authorized or z/OS removes the APF authorization of those libraries that do have the authorization. See the code example in the figure "crastart.endevor.conf - CRASTART with CA Endevor® SCM RAM" in chapter "(Optional) Common Access Repository Manager (CARMA)", section "CRASTART with CA Endevor® SCM RAM", subsection "Customize crastart.endevor.conf" of the IBM Rational Developer for System z Host Configuration Guide (SC23-7658). This is where you customize the file crastart.endevor.conf. There is a line

TASKLIB = dataset1,dataset2,... 

All of those data sets must be APF authorized.

Additional information

None.

ERROR com.ibm.carma.rse.server.miners.CARMAMiner: CARMA error occured RDZ - ENDEVOR error 500

Problem

Attempts to install or set up CARMA result in this error message when using IBM Rational Developer for System z.

Error

ERROR com.ibm.carma.rse.server.miners.CARMAMiner: 
  CARMA error occured RDZ - ENDEVOR error 500
ERROR com.ibm.carma.rse.server.miners.CARMAMiner: CARMA error occured 
  during command processing
com.ibm.carma.rse.server.transport.CARMAException: 500: 

Explanation and resolution

The 500 error code is returned from Endevor, so you are already in Endevor.

MSG3FILE might be empty because CA Endevor never gets past an access error. Check BSTERR in case it contains the information that could be key to solve this problem.

Additional information

None.

CARMA server is not starting up in timely manner

Problem

The CARMA server is not starting up in timely manner.

Error

com.ibm.carma.CARMAException: CRRZC2200E An error was received from the
  CARMA Host [E09Z]:
[406] 406: Server could not be loaded in appropriate timeframe. Contact
  system programmer.
at com.ibm.carma.client.subsystem.CARMASubSystem.processErrorMessage
(Unknown Source)   

Explanation and resolution

Rational Developer for System z cannot connect to CARMA.

Possible causes:

  1. Wrong comment sign used in crastart.endevor.conf. * is needed , # was used.
  2. CRASTART in /usr/lpp/rdz/bin does not have the sticky bit active.
  3. FEK.SFEKLPA(CRASTART) is not in LPA.
  4. Task list data sets in crastart.endevor.conf are not both APF authorized or are not in the link list.

Solutions:

  1. Correct the comment sign used.
  2. Make sure sticky bit is set.
  3. Make sure CRASTART is in LPA.
  4. Make sure both SKEFLOAD and Endevor LOADLIB are APF-authorized, or comment out TASKLIB and have both in the LINKLIST and Endevor LOADLIB APF-authorized. (This is a prerequisite.)

Additional information

None.

Notices IBM Rational Developer for System z

Copyright IBM Corporation - 2012

U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

Programming interfaces: Intended programming interfaces allow the customer to write programs to obtain the services of .

IBM Director of Licensing
IBM Corporation
North Castle Drive
Armonk, NY 10504-1785
U.S.A.

For license inquiries regarding double-byte (DBCS) information, contact the IBM Intellectual Property Department in your country or send inquiries, in writing, to:

Intellectual Property Licensing
Legal and Intellectual Property Law
IBM Japan, Ltd.
3-2-12, Roppongi, Minato-ku, Tokyo 106-8711 Japan

The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.

Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you.

Licensees of this program who wish to have information about it for the purpose of enabling: (i) the exchange of information between independently created programs and other programs (including this one) and (ii) the mutual use of the information which has been exchanged, should contact:

Intellectual Property Dept. for Rational Software
IBM Corporation
20 Maguire Road
Lexington, Massachusetts 02421-3112
U.S.A.

Such information may be available, subject to appropriate terms and conditions, including in some cases, payment of a fee.

The licensed program described in this document and all licensed material available for it are provided by IBM under terms of the IBM Customer Agreement, IBM International Program License Agreement or any equivalent agreement between us.

Any performance data contained herein was determined in a controlled environment. Therefore, the results obtained in other operating environments may vary significantly. Some measurements may have been made on development-level systems and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurements may have been estimated through extrapolation. Actual results may vary. Users of this document should verify the applicable data for their specific environment.

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.

All statements regarding IBM's future direction or intent are subject to change or withdrawal without notice, and represent goals and objectives only.

This information is for planning purposes only. The information herein is subject to change before the products described become available.

This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental.

Copyright license

This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. The sample programs are provided "AS IS", without warranty of any kind. IBM shall not be liable for any damages arising out of your use of the sample programs.

Each copy or any portion of these sample programs or any derivative work, must include a copyright notice as follows:

If you are viewing this information softcopy, the photographs and color illustrations may not appear.

Trademark acknowledgments

IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at www.ibm.com/legal/copytrade.shtml.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates.

Other product and service names might be trademarks of IBM or other companies.


Terms of use | Feedback

This information center is powered by Eclipse technology. (http://www.eclipse.org)