A bitcoin transaction typically looks as follows.

Source: Blockchain.com

Transaction Size

For each transaction there are,

180 (input) +  34 (output) + 10 (extra) = 224 bytes

If input is compressed 180 - 32 = 148 bytes

You can find the formula here

In the simplest case, you'll have one input and two outputs (the recipient, and change). So in=1 and out=2. According to the linked formula, the transaction size will be about

180 + 2 * 34+10 = 258 bytes

Transaction Fee

Today you can see most transactions with 1-10 satoshi/Byte.

1 satoshi = 0.00000001 BTC

If the transaction fee is 10 satoshi/Byte, then for transferring 258 Bytes of transaction, 0.00002580 BTC needs to paid.

258 * 10 = 2580 satoshi = 0.00002580 BTC

Even if we have more inputs and outputs, we can say the transaction fee should be typically 0.0001 BTC. And you shouldn't pay more than that. BTC/USD rates are often fluctuating. Let us consider the price of 1 BTC = 10,000 USD. In this case, we pay 1 USD as a fee. You can always set the free around this amount for your transaction to go through.

The transaction fee doesn't depend on the amount being transferred. It depends on the amount of data being transferred.

If we increaes the inputs (Use multiple source account in the transaction) or use multiple outputs (typically exchanges and pools do this, to transfer funds to different addresses), the transaction size increases. This transaction has two inputs and 2 outputs are provided.

Now let us understand how the transaction fee is decided. And how it is connected to bitcoin mining.

Understanding the Block Mining Fee

The block size is limited to 1 MB. So a miner has to choose transactions to fit in 1 MB of size. Each transaction comes with its own transaction fee.

Don't confuse block mining fee with the block reward of 12.5 BTC.

Let us say, a transaction pays 0.01 BTC for it to be included in the block. And another transaction pays 0.015 BTC. And they are 512 KB each. So the miner gets 0.01 + 0.015 = 0.025 BTC for mining this block.

Another case can be, 4 transactions pay 0.01 BTC each with the size of 256 KB (256 * 4 = 1MB), now the total fee for the block will go up to 0.04 BTC. So the miner will profit more by including these smaller 4 transaction to improve his mining fee.

Here we understand that,

The Transaction fee in BTC does not matter to the miner, satoshi/Byte matters to him. Miners try to maximize the fee evaluating (Transaction Fee/Transaction Size). It is similar to knapsack problem.

Here are the three examples below.

0.01 BTC/512 KB = 0.00001950 BTC/KB = 1950 sat/KB = 1.95 satoshi/Byte
((0.015/512)/1000) * 10^8 = 2.92 satoshi /Byte
(0.01/256)* 10^5 = 3.9 satoshi /Byte

The miner uses this calculation before adding a transaction to the block.

Transaction fee and confirmation delay

If the miner successfully mines a block, he gets all the transaction fees for the transactions he included in his block. So naturally, he will pick the transactions with the highest transaction fee.

Your transaction will not be confirmed unless it is included in a block. If you pay lower satoshi/Byte than what most people are paying, your transaction will not be picked up from the pool till better-paying transactions are there for miner.

Organizing transactions from an unconfirmed transaction pool into a block(1MB) with a maximum fee, makes the mining fee go up.

The miners don't pick up transactions with less fees. And those transactions are picked when higher fee transactions are not available in the unconfirmed pool. That is why we see delays in days for such transactions to confirm.

Reducing Transaction Fees with batching

Transaction batching is the process of combining multiple transactions into one transaction rather than creating a new transaction for every payment. It drastically reduces the costs of transacting bitcoin and reduces the overall Bitcoin network load, allowing it to fit more in a smaller space.

Transaction batching is used by exchanges to handle payouts for customers. To move funds outside the exchange.

In addition to payment batching directly providing a fee savings, batching also uses the limited block space more efficiently by reducing the block space.

If you combined 10 payments into one transaction, rather than sending them individually, you could save 75% of the block space.

Why the fees were so high in Jan 2018?

It is said a group of people wanted the bitcoin to become useless. So they inflated the transaction fee. How is that possible. They simply paid a large fee for the transaction.

So let us say someone pays 0.1 BTC for the transaction of size 256 KB. So fee becomes 39 satoshi/Byte. But still, this individual is letting in 4 other such transactions by paying 3.9 satoshi/Byte. So he adds a lot of unnecessary signatures to the input, making the size go up to 1 MB. Still paying a higher fee than others

(0.1/1024)* 10^5 = 9.7 satoshi/Byte and filling up all the space in the block.

Look at this transaction. And there are many other such transactions this block. Total of only 78 transactions.