<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Observability on Kinoko's TIL Log</title><link>https://kinoko-tech-blog-theta.vercel.app/tags/observability/</link><description>Recent content in Observability on Kinoko's TIL Log</description><generator>Hugo -- gohugo.io</generator><language>en</language><managingEditor>pippimotta@gmail.com (Kinoko)</managingEditor><webMaster>pippimotta@gmail.com (Kinoko)</webMaster><copyright>&amp;copy; Brewed by Kinoko</copyright><lastBuildDate>Tue, 17 Mar 2026 00:00:00 +0000</lastBuildDate><atom:link href="https://kinoko-tech-blog-theta.vercel.app/tags/observability/index.xml" rel="self" type="application/rss+xml"/><item><title>Error &amp; Log by Middleware</title><link>https://kinoko-tech-blog-theta.vercel.app/posts/error-log-by-middleware/</link><pubDate>Tue, 17 Mar 2026 00:00:00 +0000</pubDate><author>pippimotta@gmail.com (Kinoko)</author><guid>https://kinoko-tech-blog-theta.vercel.app/posts/error-log-by-middleware/</guid><description>&lt;h2 id="the-point"&gt;The Point&lt;/h2&gt;
&lt;p&gt;Each request produces exactly one log entry, handled by middleware. Errors are only wrapped at boundaries with third-party code. Logging follows the OpenTelemetry standard so logs can plug into any observability backend.&lt;/p&gt;
&lt;h2 id="explanation"&gt;Explanation&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;Four core principles&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;1. Every request gets exactly one log&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Instead of logging in every handler or function, middleware outputs a single structured log entry when the request completes, containing status code, latency, request ID, and other fields. This avoids scattered log entries for the same request that are hard to correlate.&lt;/p&gt;</description><content:encoded><![CDATA[<h2 id="the-point">The Point</h2>
<p>Each request produces exactly one log entry, handled by middleware. Errors are only wrapped at boundaries with third-party code. Logging follows the OpenTelemetry standard so logs can plug into any observability backend.</p>
<h2 id="explanation">Explanation</h2>
<p><strong>Four core principles</strong></p>
<p><strong>1. Every request gets exactly one log</strong></p>
<p>Instead of logging in every handler or function, middleware outputs a single structured log entry when the request completes, containing status code, latency, request ID, and other fields. This avoids scattered log entries for the same request that are hard to correlate.</p>
<p><strong>2. Log should be wrapped in a proper way</strong></p>
<p>Logs should be structured (structured logging) &ndash; not <code>fmt.Println(&quot;error:&quot;, err)</code>, but a format with fields:</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json"><span class="line"><span class="ln">1</span><span class="cl"><span class="p">{</span>
</span></span><span class="line"><span class="ln">2</span><span class="cl">  <span class="nt">&#34;level&#34;</span><span class="p">:</span> <span class="s2">&#34;error&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="ln">3</span><span class="cl">  <span class="nt">&#34;request_id&#34;</span><span class="p">:</span> <span class="s2">&#34;abc-123&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="ln">4</span><span class="cl">  <span class="nt">&#34;method&#34;</span><span class="p">:</span> <span class="s2">&#34;POST&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="ln">5</span><span class="cl">  <span class="nt">&#34;path&#34;</span><span class="p">:</span> <span class="s2">&#34;/orders&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="ln">6</span><span class="cl">  <span class="nt">&#34;latency_ms&#34;</span><span class="p">:</span> <span class="mi">42</span><span class="p">,</span>
</span></span><span class="line"><span class="ln">7</span><span class="cl">  <span class="nt">&#34;error&#34;</span><span class="p">:</span> <span class="s2">&#34;calling payment sdk: connection refused&#34;</span>
</span></span><span class="line"><span class="ln">8</span><span class="cl"><span class="p">}</span></span></span></code></pre></div><p><strong>3. Only wrap errors on boundaries with 3rd party code</strong></p>
<p>A <strong>boundary</strong> is the seam between your code and a system you do not own &ndash; calling a DB, calling a third-party SDK, calling an external API are all boundaries.</p>
<p>Errors from third parties lack your context, so wrap once at the boundary. Between internal functions, just pass the error up.</p>
<p>Given the chain: <code>Handler -&gt; getOrder() -&gt; queryOrderFromDB()</code>:</p>





<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="ln"> 1</span><span class="cl"><span class="c1">// Layer 3: DB boundary, the only place to wrap</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="kd">func</span><span class="w"> </span><span class="nf">queryOrderFromDB</span><span class="p">(</span><span class="nx">ctx</span><span class="w"> </span><span class="nx">context</span><span class="p">.</span><span class="nx">Context</span><span class="p">,</span><span class="w"> </span><span class="nx">id</span><span class="w"> </span><span class="kt">int</span><span class="p">)</span><span class="w"> </span><span class="p">(</span><span class="o">*</span><span class="nx">Order</span><span class="p">,</span><span class="w"> </span><span class="kt">error</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="w">    </span><span class="nx">row</span><span class="p">,</span><span class="w"> </span><span class="nx">err</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">db</span><span class="p">.</span><span class="nf">Query</span><span class="p">(</span><span class="nx">ctx</span><span class="p">,</span><span class="w"> </span><span class="s">&#34;SELECT * FROM orders WHERE id = ?&#34;</span><span class="p">,</span><span class="w"> </span><span class="nx">id</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="w">    </span><span class="k">if</span><span class="w"> </span><span class="nx">err</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="kc">nil</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="w">        </span><span class="c1">// Wrap at the boundary, describe what operation was happening</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="w">        </span><span class="k">return</span><span class="w"> </span><span class="kc">nil</span><span class="p">,</span><span class="w"> </span><span class="nx">fmt</span><span class="p">.</span><span class="nf">Errorf</span><span class="p">(</span><span class="s">&#34;querying order id=%d: %w&#34;</span><span class="p">,</span><span class="w"> </span><span class="nx">id</span><span class="p">,</span><span class="w"> </span><span class="nx">err</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="w">    </span><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="w">    </span><span class="c1">// ...</span><span class="w">
</span></span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="c1">// Layer 2: internal function, pass up directly, do not re-wrap</span><span class="w">
</span></span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="kd">func</span><span class="w"> </span><span class="nf">getOrder</span><span class="p">(</span><span class="nx">ctx</span><span class="w"> </span><span class="nx">context</span><span class="p">.</span><span class="nx">Context</span><span class="p">,</span><span class="w"> </span><span class="nx">id</span><span class="w"> </span><span class="kt">int</span><span class="p">)</span><span class="w"> </span><span class="p">(</span><span class="o">*</span><span class="nx">Order</span><span class="p">,</span><span class="w"> </span><span class="kt">error</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="w">    </span><span class="nx">order</span><span class="p">,</span><span class="w"> </span><span class="nx">err</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nf">queryOrderFromDB</span><span class="p">(</span><span class="nx">ctx</span><span class="p">,</span><span class="w"> </span><span class="nx">id</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="w">    </span><span class="k">if</span><span class="w"> </span><span class="nx">err</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="kc">nil</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="ln">15</span><span class="cl"><span class="w">        </span><span class="c1">// Just return, no new wrap</span><span class="w">
</span></span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="w">        </span><span class="k">return</span><span class="w"> </span><span class="kc">nil</span><span class="p">,</span><span class="w"> </span><span class="nx">err</span><span class="w">
</span></span></span><span class="line"><span class="ln">17</span><span class="cl"><span class="w">    </span><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="ln">18</span><span class="cl"><span class="w">    </span><span class="k">return</span><span class="w"> </span><span class="nx">order</span><span class="p">,</span><span class="w"> </span><span class="kc">nil</span><span class="w">
</span></span></span><span class="line"><span class="ln">19</span><span class="cl"><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="ln">20</span><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="ln">21</span><span class="cl"><span class="c1">// Layer 1: handler, also does not wrap -- let middleware handle the log</span><span class="w">
</span></span></span><span class="line"><span class="ln">22</span><span class="cl"><span class="kd">func</span><span class="w"> </span><span class="nf">handleGetOrder</span><span class="p">(</span><span class="nx">w</span><span class="w"> </span><span class="nx">http</span><span class="p">.</span><span class="nx">ResponseWriter</span><span class="p">,</span><span class="w"> </span><span class="nx">r</span><span class="w"> </span><span class="o">*</span><span class="nx">http</span><span class="p">.</span><span class="nx">Request</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="ln">23</span><span class="cl"><span class="w">    </span><span class="nx">order</span><span class="p">,</span><span class="w"> </span><span class="nx">err</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nf">getOrder</span><span class="p">(</span><span class="nx">r</span><span class="p">.</span><span class="nf">Context</span><span class="p">(),</span><span class="w"> </span><span class="nx">id</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="ln">24</span><span class="cl"><span class="w">    </span><span class="k">if</span><span class="w"> </span><span class="nx">err</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="kc">nil</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="ln">25</span><span class="cl"><span class="w">        </span><span class="c1">// Hand off to middleware, do not log here</span><span class="w">
</span></span></span><span class="line"><span class="ln">26</span><span class="cl"><span class="w">        </span><span class="nx">http</span><span class="p">.</span><span class="nf">Error</span><span class="p">(</span><span class="nx">w</span><span class="p">,</span><span class="w"> </span><span class="s">&#34;internal error&#34;</span><span class="p">,</span><span class="w"> </span><span class="nx">http</span><span class="p">.</span><span class="nx">StatusInternalServerError</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="ln">27</span><span class="cl"><span class="w">        </span><span class="k">return</span><span class="w">
</span></span></span><span class="line"><span class="ln">28</span><span class="cl"><span class="w">    </span><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="ln">29</span><span class="cl"><span class="w">    </span><span class="c1">// ...</span><span class="w">
</span></span></span><span class="line"><span class="ln">30</span><span class="cl"><span class="p">}</span></span></span></code></pre></div><p><strong>If every layer wraps, the error message becomes:</strong></p>





<pre tabindex="0"><code>handleGetOrder: getOrder: queryOrderFromDB: querying order id=42: connection refused</code></pre><p>The repeated function names add no new information &ndash; just noise.</p>
<p><strong>Wrapping only at the boundary is much clearer:</strong></p>





<pre tabindex="0"><code>querying order id=42: connection refused</code></pre><p>The rule: <strong>wrap = add new information. No new information, just return err.</strong></p>
<p><strong>4. OTEL compliant logging in middleware</strong></p>
<p>OTEL = <strong>OpenTelemetry</strong>, the CNCF observability standard that defines a unified format and API for traces, metrics, and logs.</p>
<p>When middleware outputs OTEL-compliant logs, you can plug into any backend (Grafana, Datadog, GCP Cloud Logging) without changing application code.</p>





<pre tabindex="0"><code>Request → Middleware (start timer, inject trace context)
           ↓
         Handler (process logic, pass errors up)
           ↓
         Middleware (at the end, log one entry with status / latency / error)</code></pre><h2 id="knowledge-sugar">Knowledge Sugar</h2>
<p><strong>Why not let each handler log on its own?</strong></p>
<p>Problems with scattered logging:</p>
<ul>
<li>The same request might get logged multiple times across functions &ndash; hard to correlate</li>
<li>Inconsistent formats &ndash; hard to filter with queries</li>
<li>Duplicate logs &ndash; more noise, higher cost</li>
</ul>
<p>Middleware-centralized logging means: one request = one log entry = one trace. Searching and debugging become much more intuitive.</p>
<p><strong>Error wrap decision guide</strong></p>
<table>
	<thead>
			<tr>
					<th>Location</th>
					<th>Wrap?</th>
					<th>Reason</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>Calling DB / third-party SDK</td>
					<td>Yes</td>
					<td>Their errors lack your context</td>
			</tr>
			<tr>
					<td>Calling another internal service</td>
					<td>Usually yes</td>
					<td>Cross-service boundary</td>
			</tr>
			<tr>
					<td>Passing between internal functions</td>
					<td>No</td>
					<td>Just stacks messages with no new information</td>
			</tr>
	</tbody>
</table>
]]></content:encoded></item></channel></rss>