The two keys are mathematically related to each other, such that the same formula that encrypts a message (turns one big number into another) with one key can decrypt the message (turn a big number back into the original big number) with the other key. There's lots of exponents and modulo math going on but it makes sense once you strip out the computer science and math theory.
Take prime numbers P and Q, and multiply them to create rather large number N, which will be common to both the public and private keys.
Pick prime number E so that it is less than the least common multiple of P-1 and Q-1 (and also not a factor).
Calculate D based on E, P and Q. (I'm skipping the math, see Wikipedia.)
So now we have the very large number N, prime number E, and D which is magically related to E.
Let N and E be the public key, and N and D be the private key.
To encrypt plaintext message M, treat it as a number and raise it to the Eth power, and modulo it against N (repeatedly subtract N and keep the remainder), resulting in cryptogram C.
To decrypt cryptogram C, raise it to the Dth power and modulo it against N, resulting in the plaintext M.