BN_hex2bn magically segfaults in openSSL
- by xunil154
Greetings, this is my first post on stackoverflow, and i'm sorry if its a bit long.
I'm trying to build a handshake protocol for my own project and am having issues with the server converting the clients RSA's public key to a Bignum. It works in my clent code, but the server segfaults when attempting to convert the hex value of the clients public RSA to a bignum.
I have already checked that there is no garbidge before or after the RSA data, and have looked online, but i'm stuck.
header segment:
typedef struct KEYS {
RSA *serv;
char* serv_pub;
int pub_size;
RSA *clnt;
} KEYS;
KEYS keys;
Initializing function:
// Generates and validates the servers key
/* code for generating server RSA left out, it's working */
//Set client exponent
keys.clnt = 0;
keys.clnt = RSA_new();
BN_dec2bn(&keys.clnt->e, RSA_E_S); // RSA_E_S contains the public exponent
Problem code (in Network::server_handshake):
// *Recieved an encrypted message from the network and decrypt into 'buffer' (1024 byte long)*
cout << "Assigning clients RSA" << endl;
// I have verified that 'buffer' contains the proper key
if (BN_hex2bn(&keys.clnt->n, buffer) < 0) {
Error("ERROR reading server RSA");
}
cout << "clients RSA has been assigned" << endl;
The program segfaults at
BN_hex2bn(&keys.clnt->n, buffer)
with the error (valgrind output)
Invalid read of size 8
at 0x50DBF9F: BN_hex2bn (in /usr/lib/libcrypto.so.0.9.8)
by 0x40F23E: Network::server_handshake() (Network.cpp:177)
by 0x40EF42: Network::startNet() (Network.cpp:126)
by 0x403C38: main (server.cpp:51)
Address 0x20 is not stack'd, malloc'd or (recently) free'd
Process terminating with default action of signal 11 (SIGSEGV)
Access not within mapped region at address 0x20
at 0x50DBF9F: BN_hex2bn (in /usr/lib/libcrypto.so.0.9.8)
And I don't know why it is, Im using the exact same code in the client program, and it works just fine. Any input is greatly appriciated!
