2194396 – After Hana upgrade or SSFS key change wrong SSFS key on System Replication secondary site

After upgrading the HSR secondary site to SAP HANA Revision 101 or Revison 97.01 or SSFS key change on primary site, the system may not start and the following message is written into the trace file:
RootKeyStore.cpp(00272) : RSecSSFs: SSFS-3600: Record “HDB_SERVER/PERSISTENCE/ROOTKEY” was inserted with an encryption key that was different from the current one; when you still know the old one, you can try the “migrate” operation of the “rsecssfx” utility [/sapmnt/ld7272/a/HDB/jenkins_prod/workspace/FA_CO_LIN64GCC47_rel_fa~newdb100_maint_rel/sys/src/spine/src/krn/rsec/rsecssfs.c 3757]
[66679]{-1}[-1/-1] 2015-07-15 11:04:22.160394 w Crypto           RootKeyStore.cpp(00412) : Error during reading of SSFS: exception  1: no.301103  (Crypto/SecureStore/RootKeyStore.cpp:676)
RootKeyStoreReader::read(): SSFS-3600: Record “HDB_SERVER/PERSISTENCE/ROOTKEY” was inserted with an encryption key that was different from the current one; when you still know the old one, you can try the “migrate” operation of the “rsecssfx” utility

Due to a programming error only the SSFS data file but not the corresponding key file gets updated during restart and topology reload on secondary site.
If this happens, the SSFS cannot be correctly decrypted and the message above will be written into the trace file.
The problem shows up sporadically
1.) after upgrade to SAP HANA Revision 101 or Revision 97.01
2.) or after changing the SSFS key on primary site

As a short term solution, SSFS key and data file from secondary site should be compared with the files on primary site. They can be found in the following directory: $(DIR_GLOBAL)/hdb/security/ssfs.
SSFS data file is expected to be the same for both sites, but the key files should be different in this case.

1.) If the key file exists on both sites and there is a difference, the SSFS key from primary site has to be copied to the secondary site.
2.) If there is no SSFS key available on primary, the key file on seconday site has to be deleted
When the problem has been resolved, it can re-appear only after the SSFS key has been changed again on primary site.
The programming error will be corrected in one of the upcoming revisions.