From a simplified (yeah, this is simplified) technical perspective:
Bitcoin relies on two computational principles: hashing and public key cryptography.
Hashing: Hashing simply takes a string of letters/numbers/binary data and converts it to a number with a fixed range. For example, a (very bad) hash might be to take all the numbers, add them up, and take the ones digit. So the hash of 1234567 is 8. A good hash function has several important properties. The first two are that they should be fast to compute and that you always get the same answer (if you need randomness, it must be placed in the message before computing the hash). The hash function should also not be invertible (I cannot figure out your message is 1234567 from the hash of "8")8). The hash function should also give a very different answer for a slightly changed input. Finally, I should not be able to easily find another input into the hash function that gives the same output. Easily is defined as anything simpler than a brute force search of all possible inputs.
My example hash fails on the last two properties. The number 12345670 gives me the same hash value even though I added a 0 to the end, and I can also give you the number 44 to get the same hash. Good hash functions are developed by teams of mathematicians and undergo rigorous mathematical proofs of their properties.
The invertibility criteria is not just a function of the hash function, but also of how it is used. For example, if I want to make a prediction of who wins the Gator Bowl, but I don't want to reveal it to you until after the game is over, I can give you the SHA-256 (SHA means Secure Hash Algorithm, and 256 is the number of bits in the output hash) hash of my prediction, which is 003e761d9dbff59ed59224db58962d8600c3521729fe6ec8ead0c196ac912aca. The only problem is that the input space I used was small. Namely, you can try a few variations of team names (mississippi state, msu, M-State, Mississippi State, Lousiville, UL, louisville, etc.) and you'll eventually find my answer. The solution is to add a little bit of randomness to my answer. I can put in Mississippi State-XXXXXXXXXXXX or Lousiville-XXXXXXXXXXXX where the 12 Xs are a 12 random digits, I can hash the answer and you will have to brute force through around 1 trillion combinations to get my prediction. In reality, I can put way more than 12 digits on my prediction at a very low upfront cost and make my prediction uncrackable. Once the game is over, and I want to prove I was right, I provide you with Mississippi State-867930519417, you hash the string, and see that I was correct. The final hash property (not being able to easily find another input that gives the same output) means that I would not feasibly be able to find an input of Louisville-XX...
What does this have to do with Bitcoin? Suppose you have a list of all of today's transactions, along with a hash value that corresponds to yesterday and days before transactions. You can concatenate all the data to get PREVIOUS_HASH++Transaction 1++Transaction 2++...+Transaction Last++Mining Transaction++NONCE. The miners have to determine what "NONCE" is. Nonce means number used once, and in this case the miners are trying to find a NONCE that when added to "PREVIOUS_HASH++Transaction 1++Transaction 2++...+Transaction Last" gives you a hash that is below a certain value. A simpler idea is to find a hash that has a certain number of leading zeros. This is the difficulty level. My hash of "Mississippi State" above had two leading zeros (each digit in the has represents 4 bits, so the first 8 bits out of 256 bits were 0s). For bitcoin, you would have to find a nonce that gives a hash with far more zeros at the beginning. Let's say the first 32 bits need to be 0. It doesn't matter what the final 226 bits are. You will have to try around 4 billion nonces to find an answer that has that many leading 0 bits. For example, Mississippi State-2847158813 has a SHA-256 hash that begins with 8 leading zeros (verify it yourself here:
http://www.xorbin.com/tools/sha256-hash-calculator).
This number, 2^32 which is around 4*10^9, is not large enough for bitcoin (this is an extremely small number in computational standards. Your phone could find a solution in less than a couple minutes). I think bitcoin is around 10^19. If you have a single computer computing 1 billion hashes per second, it would take on average 317 years to find the nonce. However, there are millions of processors constantly mining, so a nonce is found, on average, every 10 minutes. If you find the nonce, you claim your bitcoins and publish your nonce. One other thing is that someone else cannot steal your nonce and claim it for himself because one of the transactions in the block will be the coins you award to yourself for mining (the "Mining Transaction" contains your bitcoin address). Since your bitcoin address (more on that later) is different from someone else's address (and should be because if you have the same address they can steal all your coins) and a hash value will radically change with a change in input, the nonce they find will be different from yours because there will be one transaction different.
Public key cryptography:
Public key cryptography allows me to encrypt or sign data with a private key and anyone can decrypt or verify my signature with my public key. Someone else cannot generate a signature or encrypt data with my public key and have it appear as if I signed or generated the encrypted data. Public key cryptography is computationally expensive, so the only time it is used in bitcoin is in signing transactions and verifying transactions. Even then, the transaction itself is not signed, rather the hash of the transaction is signed. For bitcoin, the public key also acts as the address, which indicates who are sending and receiving bitcoins. I can generate a public key/private key pair and have someone send me bitcoin (or mine) and put the public key as the recipient address. I can later spend the coin because I have the private key required to spend it.
The crazy thing is that in bitcoin a validly recorded transaction is itself the bitcoin. If the ledger has a transaction with me as the recipient for 1 bitcoin, I have 1 bitcoin. If I want to spend that 1 bitcoin by sending it to you, I can take your address (public key), the number of bitcoin I'm sending you, and the hash of my bitcoin (that is, the hash of the previous transaction that contains my public key as the recipient). I concatenate them together, digitally sign them with my private key, and broadcast the transaction to the bitcoin network. They are entered into the ledger, which the miners are using as an input to generate their bitcoins. Once the miners have found the nonce, the transaction is final (there are possibilities of a what is called a blockchain fork, but that's beyond the discussion here). You can check the ledger and see that you are the recipient of 1 bitcoin from me. If I ever try to spend that bitcoin again, the hash of the bitcoin I am trying to spend can be checked against the ledger and you can tell that I am trying to double spend and reject the transaction.
I can combine multiple bitcoin inputs and/or generate multiple bitcoin outputs. For example, if I have two 1-bitcoin hashes and want to send you 2 bitcoin, your transaction will have two input hashes and the resulting transaction will indicate you have 2 bitcoin. If I have a 2-bitcoin transaction and want to send you 1 bitcoin, I send you 1 bitcoin and send myself 1 bitcoin; it's a single transaction have the same input hash, but there are two outputs, one for 1 bitcoin to your address and one for 1 bitcoin to my address, and I can verify that the sum of the outputs is less than the sum of the inputs so it is accepted into the network. If I do something like send you 1 bitcoin and forget to send myself the remaining bitcoin, the remaining bitcoin becomes part of the transaction fee and miners can claim it. If I send you 2 bitcoin from a 1 bitcoin block, the network will reject the transaction.
Lost bitcoins can never be reclaimed, or could only be reclaimed if you can generate a private key corresponding to the public key of the lost bitcoin. If someone comes up with a feasible way of doing this, it will be the end of bitcoin unless a new public key system is developed.