{"componentChunkName":"component---src-templates-blog-post-jsx","path":"/blog/debugging-the-wifi-crash-that-froze-my-homelab/","result":{"data":{"site":{"siteMetadata":{"name":"Huzaifa Rasheed","title":"Huzaifa Rasheed","description":"Software Engineer","about":"\n      Hey, I'm Huzaifa.\n      <br/><br/>\n      Engineer by trade, builder by instinct - I believe in owning my stack, shipping fast, and occasionally running on chai (tea) and stubbornness. I work best in that sweet spot between deep focus and fast feedback - solo or in sync with a good team.\n      <br/><br/>\n      This site's my little corner of the internet - part portfolio, part lab - where I document what I build, break, or learn.\n      <br/><br/>\n      Outside work, I'm into long walks, pixel-perfect headshots in FPS games (eventually... maybe), plants I probably overwater, and the occasional \"classified\" hobby to stay curious.\n      <br/><br/>\n      Reach out anytime - my digital door's always open. 👋\n    ","twitter":"https://twitter.com/huzRasheed","github":"https://github.com/huzaifa-99","linkedin":"https://www.linkedin.com/in/huzaifa-rasheed/","devto":"https://dev.to/huzaifa99","stackoverflow":"https://stackoverflow.com/users/12579290/huzaifa","leetcode":"https://leetcode.com/rhuzaifa","discord":"https://discordapp.com/users/rhuzaifa","email":"dev@rhuzaifa.com","projects":[{"name":"FFMpeg Web","description":"An experimental browser-based terminal that runs FFmpeg using WebAssembly, enabling media processing directly in the browser.","link":"https://ffmpeg-web.rhuzaifa.com/","github":"https://github.com/huzaifa-99/ffmpeg-web"},{"name":"Feed base 2","description":"A mini browser game where players manipulate 4-bit binary blocks to match target BCD values - part puzzle, part binary logic trainer.","link":"https://feedbase2.rhuzaifa.com/","github":"https://github.com/huzaifa-99/feed-base-2"},{"name":"Fabric browser extension","description":"A Chrome extension that injects engineered Fabric prompts directly into the ChatGPT interface for enhanced workflow automation.","link":"https://github.com/huzaifa-99/fabric-browser-extension","github":"https://github.com/huzaifa-99/fabric-browser-extension"},{"name":"Pure Cinema","description":"An experimental, tongue-in-cheek text-to-video generator that stitches together footage, synthesized voiceovers, and background music with a Node.js + ffmpeg pipeline. Not quite Hollywood, but it renders.","link":"https://cinema.rhuzaifa.com","github":null},{"name":"Aria2c Packload","description":"A Bash script for bulk downloading magnet links or torrents using aria2c - optimized for series or list-based transfers.","link":"https://github.com/huzaifa-99/aria2c-packload","github":"https://github.com/huzaifa-99/aria2c-packload"},{"name":"RSS Watchdog","description":"A lightweight Bash script that watches RSS/Atom feeds and compiles a Markdown-based reading checklist for Unix systems.","link":"https://github.com/huzaifa-99/rss-watchdog","github":"https://github.com/huzaifa-99/rss-watchdog"},{"name":"QuoteGen","description":"A quote graphic generator that produces stylized quote images with random selection and a built-in editor for customization.","link":"https://quotegen.rhuzaifa.com/","github":null},{"name":"WebRTC Video Chat","description":"A basic WebRTC-powered app enabling peer-to-peer video and audio calls between two users.","link":"https://webrtc-video-chat.rhuzaifa.com/","github":null}],"experience":null,"skills":[{"name":"Languages & Frameworks","description":"JavaScript, TypeScript, Python, Bash - Frameworks include Node.js, React, Next.js, Vue, React Native, FastAPI."},{"name":"Databases & Storage","description":"PostgreSQL, MySQL, MongoDB - Experience with schema design, indexing, query optimization, and migrations."},{"name":"Cloud & Infrastructure","description":"AWS (EC2, RDS, S3, Lambda), Vercel, Netlify, Heroku - Comfortable with serverless, autoscaling, and cost optimization."},{"name":"DevOps & Tooling","description":"Docker, Git, CI/CD pipelines (GitHub Actions, GitLab CI) - Experience with observability, containerization, and release workflows."},{"name":"Testing & QA Automation","description":"Jest, Playwright, Puppeteer, Selenium - Focus on E2E testing, mocking APIs, and maintaining test coverage."}]}},"markdownRemark":{"id":"8d66cd41-02aa-505e-b851-ebfab68daab0","excerpt":"I just wanted my homelab to run smoothly. But sometimes, the universe (and Linux) has other plans. My home server hosts a bunch of small things on my old…","html":"<p>I just wanted my homelab to run smoothly. But sometimes, the universe (and Linux) has other plans.</p>\n<p>My home server hosts a bunch of small things on my old hardware. </p>\n<p>It connects via Wi-Fi, because i didn’t want to run a 20-meter Ethernet cable halfway across the house to where my router is. </p>\n<p>So… I stuck with Wi-Fi.  </p>\n<p>Little did i know, the trouble it would cause.</p>\n<hr>\n<h2>⚡ The Mystery Crashes</h2>\n<p>Every once in a while, my entire system would just <em>freeze</em>. </p>\n<p>No response, no errors, no log, no warning. Just fans spinning and the little lights blinking.  </p>\n<p>The only way to bring it back? Pull the plug.  </p>\n<p>At first, I thought I was overloading it. Maybe too many containers, maybe overheating, maybe the RAM.  </p>\n<p>My mind was saying “it’s probably hardware”.</p>\n<p>It wasn’t until a few nights ago, when it crashed again mid-session, I decided to dig in. </p>\n<p><code class=\"language-text\">syslog</code> and <code class=\"language-text\">journalctl</code> didn’t give me anything obvious. </p>\n<p>That’s when I asked ChatGPT to pair-debug with me, and together we started narrowing things down.</p>\n<hr>\n<h2>🧩 The Clue</h2>\n<p>After a bit of back-and-forth, we found something interesting buried in <code class=\"language-text\">dmesg</code>:</p>\n<div class=\"gatsby-highlight\" data-language=\"bash\"><pre class=\"language-bash\"><code class=\"language-bash\">iwlwifi 0000:02:00.0: Device error - SW reset\niwlwifi 0000:02:00.0: Start IWL Event Log Dump: nothing <span class=\"token keyword\">in</span> log</code></pre></div>\n<br/>\n<h2>Wait… Wi-Fi?</h2>\n<p>I thought it was supposed to “just work”.</p>\n<p>Maybe with a little less throughput (no ethernet remember).</p>\n<p>Apparently, under high system load, Intel Wi-Fi chips can panic, crash the firmware, and sometimes freeze the entire system. </p>\n<hr>\n<h2>🔧 The Fix (and the Weird Lesson)</h2>\n<p>Turns out, the issue comes from aggressive power-saving and hardware crypto settings that don’t play nicely under load.  </p>\n<p>The fix was to offload some stuff from the Wi-Fi driver (<code class=\"language-text\">iwlwifi</code>).</p>\n<p>I added a few lines to <code class=\"language-text\">/etc/modprobe.d/iwlwifi.conf</code>:</p>\n<div class=\"gatsby-highlight\" data-language=\"bash\"><pre class=\"language-bash\"><code class=\"language-bash\">options iwlwifi <span class=\"token assign-left variable\">power_save</span><span class=\"token operator\">=</span><span class=\"token number\">0</span>\noptions iwlwifi <span class=\"token assign-left variable\">11n_disable</span><span class=\"token operator\">=</span><span class=\"token number\">8</span>\noptions iwlwifi <span class=\"token assign-left variable\">swcrypto</span><span class=\"token operator\">=</span><span class=\"token number\">1</span></code></pre></div>\n<br />\n<p>and ofc, i didn’t know what these do, so after finding out, here is what these do</p>\n<ul>\n<li>\n<p><code class=\"language-text\">power_save=0</code>: disables Wi-Fi power-saving features (like periodic sleep modes).</p>\n<ul>\n<li>Some Intel cards crash when the radio repeatedly toggles b/w sleep/wake states.</li>\n<li>Power-save transitions can trigger firmware “microcode errors.”</li>\n<li>Now, the Wi-Fi chip stays fully powered, uses slightly more power, but that’s fine.</li>\n</ul>\n</li>\n<li>\n<p><code class=\"language-text\">11n_disable=8</code>: disables A-MPDU aggregation, a feature that batches multiple pkts/frames into one large transmission.</p>\n<ul>\n<li>Certain Intel firmware versions crash when handling large aggregated frames.</li>\n<li>Disabling A-MPDU stops those crashes at the cost of some throughput.</li>\n<li>\n<p><strong>The Catch:</strong></p>\n<ul>\n<li>Wi-Fi still remains fast, but maximum throughput may drop a bit (e.g., 150 → 120 Mbps).</li>\n<li>Stability under sustained load improves dramatically.</li>\n</ul>\n</li>\n</ul>\n</li>\n<li>\n<p><code class=\"language-text\">swcrypto=1</code>: forces encryption/decryption of Wi-Fi traffic to happen in software (CPU) rather than the card’s onboard hardware</p>\n<ul>\n<li>Some Intel cards hardware crypto engines are buggy and can lock up under load, causing the “SW reset”</li>\n<li>Software crypto avoids that path entirely.</li>\n<li>\n<p><strong>The Catch/Win:</strong></p>\n<ul>\n<li>Very minor extra CPU use (I didn’t even notice).</li>\n<li>Slight latency improvement in exchange for slightly higher CPU work (again didn’t notice the load).</li>\n</ul>\n</li>\n</ul>\n</li>\n</ul>\n<h2>💪 The Stress Test</h2>\n<p>To see if it actually worked, I wanted to stress test it, which is where i found <a href=\"https://github.com/ColinIanKing/stress-ng\"><code class=\"language-text\">stress-ng</code></a>.</p>\n<p>I started with individual components test (for ~10mins each)</p>\n<div class=\"gatsby-highlight\" data-language=\"bash\"><pre class=\"language-bash\"><code class=\"language-bash\"><span class=\"token comment\"># cpu</span>\n<span class=\"token function\">sudo</span> stress-ng --cpu <span class=\"token number\">8</span> --timeout 600s --metrics-brief \n<span class=\"token comment\"># ram</span>\n<span class=\"token function\">sudo</span> stress-ng --vm <span class=\"token number\">2</span> --vm-bytes <span class=\"token number\">80</span>% --timeout 600s --metrics-brief \n<span class=\"token comment\"># disk, careful it can corrupt the disk</span>\n<span class=\"token function\">sudo</span> stress-ng --hdd <span class=\"token number\">2</span> --hdd-opts dsync --timeout 600s --metrics-brief\n<span class=\"token comment\"># i/o</span>\n<span class=\"token function\">sudo</span> stress-ng --io <span class=\"token number\">2</span> --timeout 600s --metrics-brief</code></pre></div>\n<br/>\nand \"stressing\" the Wi-Fi module in parallel\n<div class=\"gatsby-highlight\" data-language=\"bash\"><pre class=\"language-bash\"><code class=\"language-bash\"><span class=\"token comment\"># sends pkts as fast as the network allows</span>\n<span class=\"token function\">sudo</span> <span class=\"token function\">ping</span> -f -s <span class=\"token number\">1400</span> google.com </code></pre></div>\n<br/>\nAll of these independently didn't crash my system, so naturally the next thing was to \"stress\" everything together\n<div class=\"gatsby-highlight\" data-language=\"bash\"><pre class=\"language-bash\"><code class=\"language-bash\"><span class=\"token function\">sudo</span> stress-ng --cpu <span class=\"token number\">4</span> --vm <span class=\"token number\">2</span> --vm-bytes <span class=\"token number\">70</span>% --hdd <span class=\"token number\">1</span> --io <span class=\"token number\">2</span> --timeout 600s --metrics-brief\n<span class=\"token comment\"># plus Wi-Fi module stress in parallel</span>\n<span class=\"token function\">sudo</span> <span class=\"token function\">ping</span> -f -s <span class=\"token number\">1400</span> google.com </code></pre></div>\n<br/>\nit basically maxes out mostly everything - CPU, memory, disk, I/O and network all at once - the same kind of load that used to instantly kill my system.\n<br/>\n<p>This time… nothing crashed.</p>\n<p>No hangs, no lag, no freeze, just some fan noise.</p>\n<p>I was happy, my server handled it, and most importantly it was still working fine.</p>\n<h2>🧠 What I Learned</h2>\n<p>Before this, I never really thought about Wi-Fi drivers at the kernel level.</p>\n<p>But this bug made me realize how deep Linux’s stability goes - and how fragile things can get when firmware starts misbehaving.</p>\n<p>And the irony is, if I had just used Ethernet, I would’ve never even discovered this.</p>\n<p>Avoiding “the easy fix” lead me to a much better one.</p>\n<h2>✨ Takeaway</h2>\n<p>Running a homelab, among many things, also helps to understand how these system behaves under pressure.</p>\n<p>In my case, it helped me uncover and fix a bug that I didn’t even know existed. </p>\n<p>Technically, all bugs are like this when they are first discovered, but this bug especially was not on my bingo cards. </p>\n<hr>\n<p>All this, just because I didn’t want to run an Ethernet cable - but honestly, I’m glad I didn’t.</p>\n<style>\nul ul {\n  margin-bottom: 16px !important; /* remove spacing for all nested levels */\n}\nul p {\nmargin-bottom: 8px !important;\n}\n</style>","frontmatter":{"title":"Debugging the Wi-Fi Crash That Froze My Homelab","date":"November 03, 2025","description":"A simple server freeze led me deep into Wi-Fi firmware quirks and kernel driver settings"}}},"pageContext":{"slug":"/blog/debugging-the-wifi-crash-that-froze-my-homelab/","previous":{"fields":{"slug":"/blog/juice.log/"},"frontmatter":{"title":"juice.log"}},"next":null}},"staticQueryHashes":["2276319502"]}