You tried to publish a multicast stream or tweak your WMS station, and Windows threw NS_E_INVALID_MTU_RANGE (0XC00D1586) in your face. Annoying, but this one's actually simple.
The culprit here is almost always an MTU value outside the 576–1500 range Windows Media Services accepts. WMS won't let you push anything smaller than 576 (the IPv4 minimum reassembly buffer size) or bigger than 1500 (standard Ethernet). Anything else gets rejected at the API layer before the service even tries to bind.
Fix it in the WMS MMC snap-in
Open Windows Media Services from Server Manager (or hit wmssnap.msc). Expand your server, go to Publishing Points, and right-click the publishing point that's failing.
- Click Properties.
- Go to the Multicast tab (or Multicast Data Delivery depending on your WMS version — 2008 vs 2008 R2 verbatim differences).
- Look at the MTU field. If it's under 576, above 1500, or blank garbage, that's your problem.
- Set it to 1500 for standard Ethernet. Use 1400 if you're tunneling through VPN or across a WAN with jumbo-frame-averse gear.
- Apply, then restart the WMS service:
net stop wmserver
net start wmserverThat's it. Reload the publishing point and the error vanishes.
Fix it through the command line
If you're scripting this across a fleet, the WMS admin scripts are quicker. They live in %SystemRoot%\system32\windows media\bin.
cscript wmsadmin.vbs -setmtu "publishing_point_name" 1400Swap the name for your actual publishing point. Don't guess the syntax — run cscript wmsadmin.vbs -h first because Microsoft changed argument order between WMS 2003 and 2008.
Why this works
WMS validates the MTU at the API boundary before it touches the network stack. The error code NS_E_INVALID_MTU_RANGE is literally a range check — that's the whole error. The 576 floor comes from RFC 791: every IPv4 host must accept datagrams of at least 576 bytes. The 1500 ceiling is standard Ethernet frame payload. WMS refuses to guess what happens when you go out of bounds, so it rejects the whole configuration.
You can't just "set 9000 for jumbo frames" and expect it to work. WMS doesn't support jumbo MTUs on multicast publishing points. If you need jumbo frames for your storage network, that's a separate NIC config — leave WMS alone.
Less common causes
The value is stored in the registry and got corrupted
MTU lives under:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows Media\Server\Multicast\PublishingPoints\{GUID}\Check the MTU DWORD. If it's 0 or something silly like 999999, set it to 1500 and restart wmserver. I've seen profile imports from other servers dump garbage in here.
Group Policy is overriding your change
If you tweak the MTU, it looks fine, then the error comes back after a reboot — a GPO is rewriting it. Run gpresult /h gpreport.html and grep for WMS settings. Some streaming-oriented ADMX templates lock the MTU to a template value.
You're on a publishing point that was cloned
Cloned publishing points occasionally carry a stale MTU from a machine that had a non-standard NIC. Create a fresh publishing point and compare. Nine times out of ten the fresh one works and the clone is broken.
Third-party multicast adapters
Some older WMS plugins (certain IPTV middleware, older Cisco or Tandberg integrations) install a shim that intercepts MTU calls and passes wild values. Disable the plugin, restart WMS, and see if the error clears.
Prevention
- Standardize on 1400 or 1500. Pick one and put it in your build docs. Don't let admins freestyle.
- Back up publishing point configs with
cscript wmsadmin.vbs -exportafter any change. Restores are trivial if something gets munged. - Test multicast with
mcasttoolor WireShark before you take the stream live. Catching an MTU mismatch after a live event starts is a bad afternoon. - Document your WMS version. WMS 2008 R2 is the last one Microsoft shipped, and it went EOL with Server 2008 R2 in January 2020. If you're still running multicast WMS in production, you should already have a migration plan. The fix above buys you time, not a future.
Set it to 1500, restart the service, move on.