Key derivation in HD wallets utilizing the prolonged personal key vs hardened derivation

There’s plenty of confusion right here, principally bits and items of the entire scheme that’s Hierarchical Deterministic derivation, and at last two questions that appear to point lacking some level about it.
The reply to the primary query is No. The second query is extra attention-grabbing :

Let’s begin from prolonged keys, particularly BIP32 keys.
Like personal keys and public keys, prolonged keys might be both “personal” or “public”. I put each in quotes becaus each varieties of prolonged keys do comprise personal info.
Not less than sufficient to trace key use. This mechanism is utilized by {hardware} wallets and “watchonly” software program wallets on a PC.

An prolonged secret is only a base58 encoded serialization of some items of knowledge :

[ magic ][ depth ][ parent fingerprint ][ key index ][ chain code ][ key ]

The place key might be both a public key or a non-public key. Non-public keys are prepended with a single 0x00 byte, so the size of this blob stays the identical.
An prolonged secret is often derived by “traversing” some path, which means you’ll begin your derivation at some mother or father prolonged key, and consecutively derive youngster keys with particular indexes till you lastly derive the ultimate prolonged key within the path.
I will cease utilizing “prolonged” on this reply. Any more I will confer with an prolonged personal key as xprv and to an prolonged public key as xpub, and simply “keys” generally. Non-extended are simply “personal key” or “public key”.

An xprv or xpub’s magic are 4 bytes to point the community it belongs to: testnet or mainnet (t or x respectively), and the kind of key it’s (pub and prv respectively).
The depth is a byte that indecates how deep an xpriv or xpout is in a path, ranging from 00 because the depth of the grasp key, and incremented by one because the derivation of extra youngster keys is completed alongside the trail.
Word that up till now, the one distinction between xprv and xpub keys that I discussed is the prv or pub half within the magic.
It additionally needs to be clear that an xprv and xpub might be in the identical path, and in the identical depth. Which means that for such pair of xprv and xpub, the [ key ] a part of the may have a 32 byte personal key (prepended with one 00 byte) within the xprv, and a 33 byte public key which is the general public key which which you get from the personal key within the xprv.

A mother or father’s fingerprint are the primary 4 bytes of the hash160 of the public key of the mother or father. Which means that even when a mother or father xprv was used to derive a toddler xprv, it could have the identical mother or father fingerprint as if a mother or father xpub was used to derive a toddler xpub.
A parent-child relationship between keys signifies that they’re adjoining in a path.

A path is an n-tuple of indexes, often in base 10, separated by /. The vary of an index might be between zero and and 4294967295 (or 2^32-1), the place something in [0,2147483647] follows non-hardnened derivation, and indexes in [2147483648,4294967295] observe hardened derivation.
You may see that every half of the vary of indexes is used for a distinct methodology. We are able to say that there are two ranges. [0,2147483647] for non-hardened keys, and [0h,2147483647h] for hardened keys. The h signifies that the index (we’ll name it i) needs to be handled as i + 2147483648.
You are most likely extra more likely to see the h notation as a caret ' as an alternative, so 1' == 1h, however I do not suppose it is very fairly so I will follow h for now.

An instance of what a path seems to be like is :

m/0h/1/2h/2/1000000000

The m signifies that the important thing at this index is a grasp xprv or grasp xpub. A small m signifies that this prolonged secret is a grasp xprv, and massive M a grasp xpub. Following the earlier definitions, you may inform that m is the mother or father of the important thing at 0h, and the important thing at 2h is the kid of the important thing earlier than it at index 1.
To make this simpler to observe, we’ll annote the totally different keys within the path with letters {a..e} if we imply that these are xprvs and {A..B} if xpubs.

m / 0h / 1 / 2h / 2 / 1000000000
m   a    b   c    d   e

A path is often given with indexes in base10, however in the important thing itself they’re encoded in hex (base16), so a [ key index ] is at all times 4 bytes with zeros prepended if wanted. The depth and index of a grasp key are each alwasy zero, so 00 and 00000000, and so they can get to a most of FF and FFFFFFFF respectively.
So m and a are mother or father and youngster, and so are d and e. The depth of b is 02 and its index is 00000001, and the depth of c is 03 whereas its index is 80000002 (80000000 + 2).
The final youngster key to be derived is e. We are able to say that we adopted a path beginning at m, from it we derived the important thing a at index 0h, then from a we derived the important thing b at index 1.. and so forth. However what does it imply to derive a brand new key?

The remaining two components within the prolonged key format, the mother or father’s [ chain code ] and [ key ] are used along with what can be the kid key’s index to derive it.
That signifies that to derive c from b, we would feed some perform with b‘s chain code and key, and c‘s index.
A selected instance of our b and c can be :

b :

0488ADE4
02
5C1BD648
00000001
2A7857631386BA23DACAC34180DD1983734E444FDBF774041578E9B6ADB37C19
003C6CB8D0F6A264C91EA8B5030FADAA8E538B020F0A387421A12DE9319DC93368

c :

0488ADE4
03
BEF5A2F9
80000002
04466B9CC8E161E966409CA52986C584F07E9DC81F735DB683C3FF6EC7B1503F
00CBCE0D719ECF7431D88E6A89FA1483E02E35092AF60C042B1DF2FF59FA424DCA

The fields are ordered as within the construction above. On each, the magic says xprv, the depth is incremented between the mother or father and youngster, the fingerprint at c is the hash160 of the general public key that you’d get from the personal key at b, and b‘s index is within the first, non-hardened half of the vary whereas c‘s is the second, hardned half. Lastly the chain code and keys of every of the xprvs are encoded.

Deriving the chain code and key for c from b is completed with a course of known as CKDpriv, which suggests deriving a toddler xprv from a mother or father xprv. On this course of we used the chain code and key from b, and the index from c.
The vital level to make: We solely encoded c after deriving its chain code and key from what can be its index.

Any xprv can be utilized with CKDpriv to derive a toddler xprv at any index. The precise means CKDpriv will act on the enter will depend on the kid’s index being within the hardened vary, or the non-hardned vary. Principally, a CKDpriv perform runs an HMAC-SHA512 on the mother or father’s chain code and key, and the kid’s index. This hmac perform takes two values a key* (to not be confused with our occurences of key, will probably be refered to as hkey), and textual content.
The mother or father’s chain code is used because the hkey, whereas the textual content is made up of the mother or father’s key within the personal key type if the the kid’s index is within the hardened vary, [0h,2147483647h], and within the public key type if the index is within the non-hardened vary. It’s then concatenated with the kid’s index.

c‘s index is within the hardened vary, so CKDpriv‘s hmac-sha512 runs with the inputs:

HMAC-SHA512( 2A7857631386BA23DACAC34180DD1983734E444FDBF774041578E9B6ADB37C19,
             003C6CB8D0F6A264C91EA8B5030FADAA8E538B020F0A387421A12DE9319DC9336880000002 )

Which returns a 64 byte hash :

8F6154A0A82D0F68B9E5B586EA66D951DAAA071BEBD390097CC516285C791A6204466B9CC8E161E966409CA52986C584F07E9DC81F735DB683C3FF6EC7B1503F

The 32 bytes on the appropriate half of this hash, 04466B9C...C7B1503F grow to be the kid’s (c right here) chain code, and the 32 bytes on the left are used to “tweak”, which means simply “addition mod n” to the mother or father’s key, on this instance :

  8F6154A0A82D0F68B9E5B586EA66D951DAAA071BEBD390097CC516285C791A62
+
  3C6CB8D0F6A264C91EA8B5030FADAA8E538B020F0A387421A12DE9319DC93368
=
  CBCE0D719ECF7431D88E6A89FA1483E02E35092AF60C042B1DF2FF59FA424DCA   mod n

• I did not write the 00 prepended bytes within the keys right here as a result of that is simply including numbers, however these zero bytes are essential for the hash perform, so I purposely included them there.

Now that we have got c‘s chain code and key (in personal key type), we might wish to really encode c for it to be a usable xprv.
To get the fingerprint from b, we have to know the general public key of the key from b. Because it’s in personal key type, we’ll need to do multiplication:

CBCE0D719ECF7431D88E6A89FA1483E02E35092AF60C042B1DF2FF59FA424DCA * G
= 03501E454BF00751F24B1B489AA925215D66AF2234E3891C3B21A52BEDB3CD711C

Take the hash160 of this public key, and the returned hash is BEF5A2F9A56A94AAB12459F72AD9CF8CF19C7BBE.
The primary 4 bytes are b‘s fingerprint : BEF5A2F9.
Encoding the remainder of c is straightforward. Begin with the magic xprv since we derived a toddler xprv, increment the depth of b by one, then the fingerprint.
Subsequent c‘s index is encoded. We derived index 2h, so this is able to be 80000002, after which the brand new chain code and key that we received from CKDpriv.

That is mainly what hardened derivation is. The mother or father’s personal key and chain code are used to derive the kid key at some hardened index.
What if we wish to derive d? It is at index 2, so a non-hardened index. That is the second case of CKDpriv.

The distinction is in what’s used for the textual content parameter of the HMAC-SHA512 perform. As an alternative of utilizing the mother or father’s key in personal key type, we use the general public key type, so to derive d at index 2 from c, we first discover the general public key of c :

CBCE0D719ECF7431D88E6A89FA1483E02E35092AF60C042B1DF2FF59FA424DCA * G
= 0357BFE1E341D01C69FE5654309956CBEA516822FBA8A601743A012A7896EE8DC2

Then proceed following the identical steps because the above:

HMAC-SHA512( 04466B9CC8E161E966409CA52986C584F07E9DC81F735DB683C3FF6EC7B1503F,
             0357BFE1E341D01C69FE5654309956CBEA516822FBA8A601743A012A7896EE8DC200000002 )

                            tweak                                                            chain code
437984D45C4A2F5840C65B3DC6D7274E2859AD25D092DB032C49AA4D006A426B|CFB71883F01676F587D023CC53A35BC7F88F724B1F8C2892AC1275AC822A3EDD

* observe that 00 isn’t prepended to the textual content, since it is a public key.

  437984D45C4A2F5840C65B3DC6D7274E2859AD25D092DB032C49AA4D006A426B
+
  CBCE0D719ECF7431D88E6A89FA1483E02E35092AF60C042B1DF2FF59FA424DCA
=
  0F479245FB19A38A1954C5C7C0EBAB2F9BDFD96A17563EF28A6A4B1A2A764EF4   mod n

hash160( 0357BFE1E341D01C69FE5654309956CBEA516822FBA8A601743A012A7896EE8DC2 )

finger
   print
EE7AB90C|DE56A8C0E2BB086AC49748B8DB9DCE72

The remainder is straightforward, and we are able to encode :

d :

0488ADE4
04
EE7AB90C
00000002
CFB71883F01676F587D023CC53A35BC7F88F724B1F8C2892AC1275AC822A3EDD
000F479245FB19A38A1954C5C7C0EBAB2F9BDFD96A17563EF28A6A4B1A2A764EF4

The distinction between these two strategies of deriving youngster xprvs is delicate however vital. It permits CKDpub, which is a perform to derive youngster xpubs from a mother or father xpub. CKDpub works nearly the identical as CKDpriv‘s non-hardened derivation, but it surely does the derivation utilizing level addition, so fairly than including up integers to make youngster personal keys, we’re including up factors to make youngster public keys.
Discover how within the non-hardened derivation we used the mother or father’s public level for the HMAC-SHA512, we used the tweak because the added worth to the mother or father personal key to derive the kid personal key, particularly, we derived d‘s personal key.

To know CKDpub, it helps to first learn about one more BIP32 perform known as Neuter. It is objective is to transform an xprv to an xpub.
Let’s “run” Neuter on our xprv d. We’ll name the ensuing xpub D. Neuter does two issues to an xprv:
1. Exchange the magic from 0488ADE4 to 0488B21E (replaces xprv with xpub)
2. Replaces the personal key within the key area` with the general public level of the identical personal key

for our xprv d, the general public level is:

0F479245FB19A38A1954C5C7C0EBAB2F9BDFD96A17563EF28A6A4B1A2A764EF4 * G
= 02E8445082A72F29B75CA48748A914DF60622A609CACFCE8ED0E35804560741D29

(that is simply regular course of of personal key -> public key)

so the result’s:

D:

0488B21E
04
EE7AB90C
00000002
CFB71883F01676F587D023CC53A35BC7F88F724B1F8C2892AC1275AC822A3EDD
02E8445082A72F29B75CA48748A914DF60622A609CACFCE8ED0E35804560741D29

Now d is “neutered”, D has the general public key encoded, however see how the chain code, depth, fingerprint and index continued. The xpub D is on the similar place within the path because the xprv d. We will probably be utilizing the chain code and key (public key) for CKDpub, similar as CKDpriv with non-hardened derivation, however as for CKDpriv, we derived the kid personal key utilizing:

tweak + (mother or father personal key) = youngster personal key

for CKDpub we will probably be utilizing:

tweak*G + (mother or father public key) = youngster public key

This works as a result of mother or father public key is actually simply (mother or father personal key)*G, and youngster public key is simply (youngster personal key)*G. That’s, if we take the CKDpriv tweak equation and multiply all components by G, we get precisely the CKDpub tweak equation.
CKDpub can solely derive youngster xpub keys within the non-hardened index vary. It is because the data current within the mother or father xpub, particularly the general public key within the [ key ], solely applies to the non-hardened vary.
The place in CKDpriv we may use the personal key to know the general public key, we won’t go the opposite means. the HMAC-SHA512 spherical that makes use of public keys in CKDpriv applies to the non-hardened index vary.

Now that we now have neutered d to create the xpub D, subsequent within the path is e‘s with index 1000000000 (or 3B9ACA00), which is within the non-hardened vary, so we should always be capable to derive E the kid xpub from D utilizing CKDpub. We begin with hmac-sha512 of the mother or father chain code as hkey and mother or father key (public key) concatenated with the kid E‘s index :

HMAC-SHA512( CFB71883F01676F587D023CC53A35BC7F88F724B1F8C2892AC1275AC822A3EDD,
             02E8445082A72F29B75CA48748A914DF60622A609CACFCE8ED0E35804560741D293B9ACA00 )

                            tweak                                                            chain code
37D3E49D8ECB854CC518BBA096F46795A9707860BF0FC95E5B19278C997098D4|C783E67B921D2BEB8F6B389CC646D7263B4145701DADD2161548A8B078E65E9E

Multiply the tweak by the generator G so we are able to tweak the mother or father’s public key utilizing level addition :

37D3E49D8ECB854CC518BBA096F46795A9707860BF0FC95E5B19278C997098D4 * G
= 0327E992F68217BC3E88CFFC3FEAB475880145413CBE008DB22B496DF4E1C3F864  <- tweak*G

Add the tweak to the mother or father level. The result’s the kid’s public key :

  0327E992F68217BC3E88CFFC3FEAB475880145413CBE008DB22B496DF4E1C3F864
+
  02E8445082A72F29B75CA48748A914DF60622A609CACFCE8ED0E35804560741D29
=
  022A471424DA5E657499D1FF51CB43C47481A03B1E77F951FE64CEC9F5A48F7011

Get the paren’ts fingerprint :

hash160(02E8445082A72F29B75CA48748A914DF60622A609CACFCE8ED0E35804560741D29) = D880D7D8....

Lastly we are able to encode E :

0488B21E
05
D880D7D8
3B9ACA00
C783E67B921D2BEB8F6B389CC646D7263B4145701DADD2161548A8B078E65E9E
022A471424DA5E657499D1FF51CB43C47481A03B1E77F951FE64CEC9F5A48F7011

Neutering d to make D then deriving E, we are able to say that our path now seems to be like :

m / 0h / 1 / 2h / 2 / 1000000000
m / a  / b / c  / D / E

Or we are able to use the N() notation (for Neuter) to point out the place CKDpub was used, however I feel it is much less fairly.
m / a / b / c / N(d / e)

So to recap in your query, there are 3 totally different derivation strategies, two utilizing personal keys and one utilizing public keys :

1. CKDpriv to derive a toddler xprv at a hardened index
2. CKDpriv to derive a toddler xprv at a non-hardend index
3. CKDpub to derive a toddler xpub at a non-hardened index