Long story short, miners literally search for needles in haystacks. The haystack changes in size every 2 weeks depending on how fast or slow each needle is found. Let’s start from the beginning.
What is a node?
A node is the bitcoin software that forms the backbone of the bitcoin network. Full nodes contains the entire history of all bitcoin transactions, ~ 70Gb at this time of writing, and constantly run rules validating transactions such as; has this bitcoin been spent before, are each transactions (in bytes) is less than MAX_BLOCK_SIZE and are the transaction’s syntax and data structures correct. See the protocol rules for more info.
On a side note, there are different flavoured nodes. Core, XT, Classic and Unlimited. Why? I’ll leave that discussion to another article.
Source: http://xtnodes.com/#all_nodes
What is a miner?
A miner is a piece of code on a computer that collects all unconfirmed transactions (the coffee you bought 5 minutes ago with bitcoins), packages it up into a “block” and attempts to have it added as the next block on the block chain. Once something is on the block chain, it is regarded as verified and has “cemented” it’s place in history. I’ve left out a bit on confirmations that you can read about here.
What is the link between a node and a miner?
There are different variations but at a simplistic level, here is a workflow scenario.
Nodes collect and validate unconfirmed transactions. Ie making sure the coffee you purchased with bitcoins is legit.
Miners then grab a bunch of these unconfirmed transactions, orders them how they please (usually in order of highest fees to lowest fees) and then sends it back to the node to get it accepted onto the blockchain.
The node then checks the received block and if all the validation passes, it will relay this to other nodes and have the transactions accepted and confirmed.
That sounds easy! What’s the catch?
The catch is that there are ~5000 other miners around the world doing the same thing. How does a node know which miner to accept the block from?
This is where, metaphorically, the node chucks a needle into the haystack and says, I’ll accept the block from the first miner who finds the needle. In addition to this, you’ll get a reward of 25 BTC for your efforts. (Note the reward started at 50 BTC and halves every 4 yrs). This is what is commonly referred to as mining.
Let there be big haystacks and small haystacks
There is a rule in place that says on average, it should take ~10 minutes to find this needle. Over time as technology has progressed, some miners find the needle pretty quick using new technologies such as with magnifying glasses or specially designed sieves or even building robots to help out. There is a feedback loop in the algorithm that looks at the average time the needle is found in the last 2 weeks and adjusts the size of the haystack, making it bigger or smaller to keep the time around 10 minutes.
Is this analogous to the CPU -> GPU -> FPGA -> ASIC technology employed by miners since 2009.
Is anything achieved by searching for the needle?
Nope. Nothing. Zilch. It’s just a meaningless, tiresome, resource intensive and mundane task that is however important to determine which miner’s block to accept. This is also known as Proof Of Work (POW). Not Prisoner Of War.
The cool part
Go to blockchain.info and pick any random block. I’ll work with this one: https://blockchain.info/block/000000000000000002dc05113aa53af00538b3aa249b83e3cd118f7eef589085
Here you can see I’ve got the needle which is called the Nonce. A random number that satisfies the required mathematical equation. I’ll simulate manually how nodes confirm if the needle is in fact correct.
It is pretty straight forward once you get the hang of it. 6 fields are needed which I’ve grabbed from the above link.
Version = 4
hashPrevBlock = 00000000000000000575467ef2bd32c3557b1273cd53cf4da5eac25708853b1a
hashMerkleRoot = c6510aace49c298e6d331b06c1b57e837b428052974925ffe9e7240564358586
Time = 2016-04-14 22:29:11
Bits = 403056459
Nonce = 2653243004
The idea is that the double hash of the above data needs to be below the specified target which is represented here as the Bits field. The first step is to convert any non-hex data to hex.
Version = 00000004
hashPrevBlock = already in hex
hashMerkleRoot = already in hex
Time = 571019b7
Bits = 1806274b
Nonce = 9e25467c
This next bit is the trickiest. The order of the digits are juggled around just to keep everyone on their toes.
The version needs to be represented in Little Endian format. ie from 00000004 to 04000000. Same with the Time, Bits and Nonce field. The hashPrevBlock and hashMerkleRoot needs to have it’s order swapped as well.
00000000000000000575467ef2bd32c3557b1273cd53cf4da5eac25708853b1a
becomes: 1a3b850857c2eaa54dcf53cd73127b55c332bdf27e4675050000000000000000 and the same with the hashMerkleRoot.
Hang in there with me. We’re almost there! Let’s list out the final data to hash.
Version = 04000000
hashPrevBlock = 1a3b850857c2eaa54dcf53cd73127b55c332bdf27e46750500000000000000000
hashMerkleRoot = 868535640524e7e9ff2549975280427b837eb5c1061b336d8e299ce4ac0a51c6
Time = b7191057
Bits = 4b270618
Nonce = 7c46259e
Let’s join it altogether as a long string: 040000001a3b850857c2eaa54dcf53cd73127b55c332bdf27e4675050000000000000000868535640524e7e9ff2549975280427b837eb5c1061b336d8e299ce4ac0a51c6b71910574b2706187c46259e
Now take this string and use this neat online SHA256d to get: 859058ef7e8f11cde3839b24aab33805f03aa53a1105dc020000000000000000
The final step is to again flip this back the other way to have the zeros at the front.
0000000000000000002dc05113aa53af0538b3aa249b83e3cd118f7eef589085
000000000000000006274B000000000000000000000000000000000000000000
The target is shown below and you can see it is lower in value. You can tell because it has more leading zeros or you can also convert to decimal as well. (The target was derived from the Bits field via the formula here)
Conclusion
Miners are a very important part of any crypto currency ecosystem. They work 24 by 7 to help perform various tasks such as (but definitely not limited to) building block, nominating that their block should be the next block in the block chain by proving that they did some labour intensive work and in general, help the flow of transactions through the network. Without miners, there would be no bitcoins.
Hi, this article was very helpful in starting to learn about how mining actually works. I was hoping you can help me with your example above though.. The link to the Sha256d above no longer works. And when I take the Hex String and run it through Powershell using Sha256 twice I get the following:
908DB859E47601E4A35820285D05E477A397655F2A282329CE2EE45F5F0A4F76
Do you know what I need to do to get it to show what you got:
859058ef7e8f11cde3839b24aab33805f03aa53a1105dc020000000000000000
? Thanks!
Try this: echo -n hello |openssl dgst -sha256 -binary |openssl dgst -sha256 as an example. You want to hash the binary output of the first sha256, not the hex representation.