Redfish, IPMI & the BMC
Out-of-band server management for DGX and GPU clusters: what the BMC is, how IPMI and ipmitool work, how the Redfish REST API is organized, how to call its endpoints with curl and CLI tools, how IPMI and Redfish map to each other, and how BCM and NVIDIA tooling use them. Ends with four hands-on simulated tickets on a BCM head node. Objectives 1.3 and 5.2.
1. The big picture: in-band vs out-of-band
| In-band management | Out-of-band (OOB) management | |
|---|---|---|
| Runs where? | Inside the host OS (agents, SSH, nvidia-smi, nvsm, DCGM) | On the BMC, a separate small computer on the motherboard |
| Works when the OS is down? | No | Yes, even when the server is powered off (as long as it has standby power) |
| Network | The data / in-band management network | A dedicated BMC / OOB management network |
| Protocols | SSH, agents, APIs | IPMI (older), Redfish (modern), web UI, remote KVM |
| Typical jobs | GPU health, software, workloads | Power on/off/cycle, boot device, hardware event logs, sensors, firmware, remote console, virtual media |
2. What the BMC is
- The Baseboard Management Controller is a small SoC with its own processor, firmware/OS, memory, and network port, soldered onto the server's motherboard.
- It runs on standby power: as long as the PSUs are plugged in, the BMC is alive, even with the host "off."
- It reads sensors (temperatures, fans, voltages, PSU status), controls power, logs hardware events in the SEL (System Event Log), stores FRU inventory (serial and part numbers), and offers remote KVM (keyboard/video/mouse) and virtual media (mount an ISO remotely).
- The host can also talk to the BMC locally through an internal interface (KCS), which is how
ipmitoolworks on the host without a network (/dev/ipmi0).
3. IPMI: the classic protocol
| Fact | Detail |
|---|---|
| Full name | Intelligent Platform Management Interface (Intel-led spec; v2.0 is current, from 2004) |
| Transport | RMCP+ over UDP port 623, a binary protocol |
| Interfaces in ipmitool | -I lanplus = IPMI 2.0 over LAN (use this). -I lan = IPMI 1.5 (weaker). -I open = local in-band through the kernel driver. |
| Key concepts | SEL (event log), SDR (sensor data records), FRU (inventory), SOL (Serial over LAN console), chassis (power, boot device, identify LED), mc (the BMC itself) |
| Status | No longer developed. Still widespread, but being replaced by Redfish. |
4. ipmitool command tour
# Remote (out-of-band). Keep the password out of argv: -E reads IPMI_PASSWORD, or -f <file>
export IPMI_PASSWORD=<bmc-password>
ipmitool -I lanplus -H dgx01-bmc -U admin -E chassis power status
# Power
ipmitool ... chassis power status | on | off | cycle | reset | soft
# off = hard power off (like holding the button)
# soft = ACPI graceful shutdown request to the OS
# cycle = off, then on reset = hard reset (no power drop)
# Boot device (PXE for BCM reprovisioning)
ipmitool ... chassis bootdev pxe # next boot only
ipmitool ... chassis bootdev pxe options=persistent,efiboot
# Logs, sensors, inventory
ipmitool ... sel elist # System Event Log, with decoded timestamps
ipmitool ... sel save dgx01-sel.txt # keep a copy as evidence
ipmitool ... sel clear # (save it first!)
ipmitool ... sdr elist # all sensors with status
ipmitool ... sensor | grep -i temp
ipmitool ... fru print # serial numbers, part numbers
# The BMC itself
ipmitool ... mc info # BMC firmware version
ipmitool ... mc reset cold # reboot the BMC (the host keeps running)
ipmitool ... lan print 1 # BMC IP, MAC, VLAN on channel 1
ipmitool ... user list 1
# Console and locating the server
ipmitool ... sol activate # serial console over LAN (~. to exit)
ipmitool ... chassis identify 60 # blink the locator LED for 60 s
# Local (in-band), from the host OS. No credentials needed, requires root
sudo modprobe -a ipmi_devintf ipmi_si # -a: load both (otherwise the 2nd word is a module parameter)
sudo ipmitool lan print 1 # "what is my BMC's IP?"
ipmitool is the classic way to find or fix a BMC's network settings when you can log in to the host but can't reach the BMC: sudo ipmitool lan print 1, then lan set 1 ipaddr ....mc reset cold reboots the BMC, not the server. chassis power reset / cycle reboots the server. Don't mix them up.Which version, from where?
| You want | In-band (from the host OS) | Out-of-band (BMC) |
|---|---|---|
| BIOS/UEFI version | sudo dmidecode -t bios | Redfish Systems/{id} → BiosVersion. There's no standard ipmitool command for it |
| BMC firmware version | sudo ipmitool mc info | ipmitool ... mc info; Redfish Managers/{id} → FirmwareVersion |
| Manufacturer, product name, serial number, UUID | sudo dmidecode -t system | ipmitool ... fru print; Redfish Systems/{id} |
| Every firmware on a DGX/HGX | nvfwupd -t ip=<bmc> ... show_version talks to the BMC (see the nvfwupd guide); Redfish UpdateService/FirmwareInventory | |
mc info is the BMC's firmware, not the BIOS. Hardware-inventory questions usually want fru print (part and serial numbers) and mc info; sdr elist is sensor readings, and chassis power status is power state, not inventory. sel list and sel elist both read the System Event Log; elist decodes sensor names too.5. IPMI security weaknesses
| Weakness | Why it matters |
|---|---|
| Cipher suite 0 | "No authentication": any password is accepted if this suite is enabled |
| RAKP hash disclosure (IPMI 2.0 design flaw) | An attacker can request the salted password hash for a known username before authenticating, then crack it offline |
| UDP, binary, weak defaults | Hard to firewall finely; default credentials are common |
| Full hardware control | A compromised BMC means power control, console, virtual media: effectively physical access |
# Is cipher suite 0 enabled? One character per suite, starting at suite 0:
# X = disabled, a = up to Administrator, o = Operator, u = User
ipmitool ... lan print 1 | grep -i cipher
# RMCP+ Cipher Suites : 0,1,2,3,6,7,8,11,12,15,16,17
# Cipher Suite Priv Max : aaaaXXaaaXXaaXX <- first 'a' = suite 0 allowed as admin
ipmitool ... lan set 1 cipher_privs XaaaXXaaaXXaaXX # first char X = suite 0 off
# Proof: suite 0 must now be refused, even with a wrong password
ipmitool -I lanplus -C 0 -H dgx05-bmc -U admin -P wrong chassis power status
6. Redfish: the modern REST API
| Fact | Detail |
|---|---|
| Standard body | DMTF (Distributed Management Task Force), first release 2015 |
| Transport | HTTPS (port 443), JSON payloads, RESTful URLs |
| Data model | Schema-defined resources (OData conventions: @odata.id, @odata.type, Members collections) |
| Entry point | The service root: /redfish/v1/ (no auth needed for the root itself) |
| Why it's better | Human- and script-readable, secure transport, scales to fleets, covers modern needs (firmware updates, eventing, telemetry, composability) |
$ curl -s https://dgx01-bmc/redfish/v1/ --cacert bmc-ca.pem | jq
{
"@odata.id": "/redfish/v1/",
"RedfishVersion": "1.15.0",
"Systems": { "@odata.id": "/redfish/v1/Systems" },
"Chassis": { "@odata.id": "/redfish/v1/Chassis" },
"Managers": { "@odata.id": "/redfish/v1/Managers" },
"AccountService": { "@odata.id": "/redfish/v1/AccountService" },
"SessionService": { "@odata.id": "/redfish/v1/SessionService" },
"EventService": { "@odata.id": "/redfish/v1/EventService" },
"UpdateService": { "@odata.id": "/redfish/v1/UpdateService" },
"TaskService": { "@odata.id": "/redfish/v1/TaskService" },
...
}
/redfish/v1/ and follow the @odata.id links. Never hard-code IDs like /Systems/1: one vendor uses 1, another System.Embedded.1, DGX might use DGX. List the collection's Members first.7. The Redfish resource tree
| Resource | Represents | Typical uses |
|---|---|---|
/redfish/v1/Systems/{id} | The computer system (the host as the OS sees it): CPUs, memory, storage, boot settings, power state | Power actions (ComputerSystem.Reset), boot override (PXE), inventory |
/redfish/v1/Chassis/{id} | The physical enclosure: sensors, fans, PSUs, thermal and power data | Temperatures, fan speeds, PSU health, power draw (Thermal/Power, or the newer ThermalSubsystem/PowerSubsystem/Sensors) |
/redfish/v1/Managers/{id} | The BMC itself | BMC network settings, BMC reset, log services, virtual media, serial console |
/redfish/v1/AccountService | Users and roles | Create users, change passwords, roles (Administrator, Operator, ReadOnly) |
/redfish/v1/SessionService | Login sessions | Create a session token, list or kill sessions |
/redfish/v1/UpdateService | Firmware | FirmwareInventory, SimpleUpdate, multipart push updates |
/redfish/v1/TaskService | Long-running jobs | Track firmware updates and other async operations |
/redfish/v1/EventService | Push notifications | Subscribe a listener URL to alerts; Server-Sent Events stream |
/redfish/v1/TelemetryService | Metric reports | Periodic sensor/metric collection (where supported) |
8. HTTP methods, status codes, and ETags
| Method | Use in Redfish |
|---|---|
GET | Read a resource or collection |
PATCH | Change some properties (boot override, NTP server, user password) |
POST | Create a member (new user, session, event subscription) or invoke an Action (.../Actions/ComputerSystem.Reset) |
DELETE | Remove (log out a session, delete a user or subscription) |
PUT | Replace a whole resource (rarely used/supported) |
| Status | Meaning |
|---|---|
| 200 / 204 | Success (with / without a body) |
| 201 | Created (for example a new session; see the Location header) |
| 202 Accepted | Long-running operation started. Poll the task monitor URL in Location. |
| 400 | Bad request: wrong property name or value. Read the @Message.ExtendedInfo in the body. |
| 401 / 403 | Not authenticated / authenticated but not allowed (role too low) |
| 404 / 405 | No such resource / method not allowed on it |
| 412 | Precondition failed: your If-Match ETag is stale. GET again for the fresh ETag. |
| 428 | Precondition required: this service insists on an If-Match header for this PATCH and you sent none. |
| 415 | Unsupported media type: the body wasn't sent as Content-Type: application/json (plain curl -d sends form encoding). |
| 503 | Service busy or unavailable (BMC rebooting or overloaded). Retry later. |
If-Match: <ETag> header on PATCH to avoid overwriting someone else's change. GET the resource, read its ETag header, and send it back. Query options such as $expand, $select, and $filter exist in the spec, but support varies by BMC.9. Authentication: Basic auth vs sessions
# Method 1: HTTP Basic auth on every request (simple, fine for one-off calls)
curl -s -u "$BMC_USER:$BMC_PASS" --cacert bmc-ca.pem https://$BMC/redfish/v1/Systems
# Method 2: Session login (better for scripts making many calls)
curl -si --cacert bmc-ca.pem -X POST https://$BMC/redfish/v1/SessionService/Sessions \
-H 'Content-Type: application/json' \
-d '{"UserName":"'"$BMC_USER"'","Password":"'"$BMC_PASS"'"}'
# HTTP/1.1 201 Created
# X-Auth-Token: 4f9c... <- send this header on later requests
# Location: /redfish/v1/SessionService/Sessions/17
curl -s -H "X-Auth-Token: $TOKEN" --cacert bmc-ca.pem https://$BMC/redfish/v1/Chassis
# Log out (frees a session slot on the BMC)
curl -s -X DELETE -H "X-Auth-Token: $TOKEN" --cacert bmc-ca.pem https://$BMC/redfish/v1/SessionService/Sessions/17
X-Auth-Token header. BMCs have a limited number of sessions, so scripts that log in and never DELETE their session can lock everyone out until the sessions time out.curl -k (skip certificate verification) is common in tutorials because BMCs ship with self-signed certificates, but it allows man-in-the-middle attacks on admin credentials. In production, install proper certificates (CertificateService) or pin the BMC's CA with --cacert.10. Common tasks with curl
R="curl -s -u $BMC_USER:$BMC_PASS --cacert bmc-ca.pem -H Content-Type:application/json"
# 1. Find the system ID (don't guess it)
$R https://$BMC/redfish/v1/Systems | jq -r '.Members[]."@odata.id"'
# /redfish/v1/Systems/DGX
# 2. Power state and health
$R https://$BMC/redfish/v1/Systems/DGX | jq '{PowerState, Status}'
# { "PowerState": "On", "Status": { "State": "Enabled", "Health": "OK" } }
# 3. Which reset types does this BMC allow?
$R https://$BMC/redfish/v1/Systems/DGX | jq '.Actions."#ComputerSystem.Reset"'
# "ResetType@Redfish.AllowableValues": ["On","ForceOff","GracefulShutdown","GracefulRestart","ForceRestart","PowerCycle"]
# 4. Graceful reboot of the host
$R -X POST https://$BMC/redfish/v1/Systems/DGX/Actions/ComputerSystem.Reset \
-d '{"ResetType":"GracefulRestart"}'
# 5. PXE boot once (for example to reprovision with BCM)
$R -X PATCH https://$BMC/redfish/v1/Systems/DGX \
-d '{"Boot":{"BootSourceOverrideTarget":"Pxe","BootSourceOverrideEnabled":"Once"}}'
# 6. Temperatures and fans
$R https://$BMC/redfish/v1/Chassis/<chassis-id>/Thermal | jq '.Temperatures[] | {Name, ReadingCelsius, Status}'
# 7. Event log entries
$R https://$BMC/redfish/v1/Managers/<bmc-id>/LogServices | jq -r '.Members[]."@odata.id"'
$R https://$BMC/redfish/v1/Managers/<bmc-id>/LogServices/SEL/Entries | jq '.Members[] | {Created, Severity, Message}'
# 8. Reboot the BMC (not the host)
$R -X POST https://$BMC/redfish/v1/Managers/<bmc-id>/Actions/Manager.Reset -d '{"ResetType":"GracefulRestart"}'
| ResetType | Meaning | ipmitool equivalent |
|---|---|---|
On | Power on | chassis power on |
ForceOff | Immediate hard power off | chassis power off |
GracefulShutdown | Ask the OS to shut down cleanly | chassis power soft |
GracefulRestart | Ask the OS to reboot cleanly | (none; do it in-band) |
ForceRestart | Hard reset without removing power | chassis power reset |
PowerCycle | Off, then on | chassis power cycle |
Nmi | Send a non-maskable interrupt (for a crash dump) | chassis power diag |
ResetType@Redfish.AllowableValues and list collections first.More everyday tasks
| You want | Where it lives |
|---|---|
| Model, serial, BIOS version, CPU/memory summary | Systems/{id} |
| CPUs, DIMMs, drives | Systems/{id}/Processors, Memory, Storage |
| PCIe devices and NICs | Chassis/{id}/PCIeDevices, NetworkAdapters (some BMCs list PCIe under Systems) |
| Temperatures, fans, PSUs, power draw | Chassis/{id}/Sensors, ThermalSubsystem, PowerSubsystem (older: Thermal, Power) |
| Identify LED | Chassis/{id} LocationIndicatorActive (older: IndicatorLED) |
| BMC network, NTP, enabled protocols | Managers/{id}/EthernetInterfaces, Managers/{id}/NetworkProtocol |
| Event logs | Managers/{id}/LogServices (some logs are under Systems/{id}/LogServices) |
| Virtual media (remote ISO) | Managers/{id}/VirtualMedia (newer BMCs: Systems/{id}/VirtualMedia) |
| BIOS settings | Systems/{id}/Bios; changes go to the resource named in its @Redfish.Settings |
| Secure Boot | Systems/{id}/SecureBoot |
| Users, roles, lockout policy | AccountService, AccountService/Accounts, Roles |
| HTTPS certificates | CertificateService, Managers/{id}/NetworkProtocol/HTTPS/Certificates |
# 9. Inventory
$R https://$BMC/redfish/v1/Systems/DGX | jq '{Model, SerialNumber, BiosVersion, ProcessorSummary, MemorySummary}'
$R https://$BMC/redfish/v1/Systems/DGX/Processors | jq -r '.Members[]."@odata.id"'
$R https://$BMC/redfish/v1/Systems/DGX/Memory | jq '."Members@odata.count"'
$R https://$BMC/redfish/v1/Systems/DGX/Storage | jq -r '.Members[]."@odata.id"'
# 10. Sensors (the newer model) and paging: big collections come back a page at a time
$R https://$BMC/redfish/v1/Chassis/DGX/Sensors | jq '."Members@odata.count", ."Members@odata.nextLink"'
$R "https://$BMC/redfish/v1/Chassis/DGX/Sensors?\$skip=75"
$R https://$BMC/redfish/v1/Chassis/DGX/Sensors/<sensor> | jq '{Name, Reading, ReadingUnits, Status}'
# 11. PSUs and power draw
$R https://$BMC/redfish/v1/Chassis/DGX/PowerSubsystem/PowerSupplies | jq -r '.Members[]."@odata.id"'
$R https://$BMC/redfish/v1/Chassis/DGX/Power | jq '.PowerControl[0].PowerConsumedWatts' # older Power resource
# 12. Blink the identify LED so the datacenter tech pulls the right box
$R -X PATCH https://$BMC/redfish/v1/Chassis/DGX -d '{"LocationIndicatorActive":true}'
# older schemas: {"IndicatorLED":"Blinking"} ipmitool equivalent: chassis identify
# 13. BMC time (so SEL timestamps line up with host logs)
$R -X PATCH https://$BMC/redfish/v1/Managers/BMC/NetworkProtocol \
-d '{"NTP":{"ProtocolEnabled":true,"NTPServers":["ntp1.example.internal"]}}'
# 14. Users: list, add a read-only monitoring account, change a password, check lockout policy
$R https://$BMC/redfish/v1/AccountService/Accounts | jq -r '.Members[]."@odata.id"'
$R -X POST https://$BMC/redfish/v1/AccountService/Accounts \
-d '{"UserName":"monitor","Password":"'"$NEW_BMC_PASS"'","RoleId":"ReadOnly","Enabled":true}'
$R -X PATCH -H 'If-Match: *' https://$BMC/redfish/v1/AccountService/Accounts/<n> -d '{"Password":"'"$NEW_BMC_PASS"'"}'
$R https://$BMC/redfish/v1/AccountService | jq '{MinPasswordLength, AccountLockoutThreshold, AccountLockoutDuration}'
# 15. Save, then clear, the SEL
$R https://$BMC/redfish/v1/Managers/BMC/LogServices/SEL/Entries > sel-$(date +%F).json
$R -X POST https://$BMC/redfish/v1/Managers/BMC/LogServices/SEL/Actions/LogService.ClearLog -d '{}'
# 16. Query options: only if the service root advertises them
$R https://$BMC/redfish/v1/ | jq .ProtocolFeaturesSupported
$R "https://$BMC/redfish/v1/Systems/DGX?\$select=PowerState,Status"
$R "https://$BMC/redfish/v1/Systems?\$expand=.(\$levels=1)"
# 17. Replace the self-signed HTTPS certificate (so nobody needs curl -k)
$R -X POST https://$BMC/redfish/v1/CertificateService/Actions/CertificateService.GenerateCSR -d '{
"CertificateCollection": {"@odata.id": "/redfish/v1/Managers/BMC/NetworkProtocol/HTTPS/Certificates"},
"CommonName": "dgx01-bmc.example.internal", "Organization": "Example", "Country": "US"}' | jq -r .CSRString > bmc.csr
# sign bmc.csr with your CA, then install it:
$R -X POST https://$BMC/redfish/v1/CertificateService/Actions/CertificateService.ReplaceCertificate -d '{
"CertificateUri": {"@odata.id": "/redfish/v1/Managers/BMC/NetworkProtocol/HTTPS/Certificates/1"},
"CertificateType": "PEM", "CertificateString": "-----BEGIN CERTIFICATE-----\n...\n-----END CERTIFICATE-----"}'
Members@odata.nextLink is present, follow it (or use $skip) or your script silently misses entries. Members@odata.count is the total, not the page size.GenerateCSR, the certificate collection path, and whether $select/$expand work all vary by BMC. Check ProtocolFeaturesSupported and the action's @Redfish.ActionInfo before scripting.11. Firmware updates and tasks
# What firmware is installed?
$R https://$BMC/redfish/v1/UpdateService/FirmwareInventory | jq -r '.Members[]."@odata.id"'
$R https://$BMC/redfish/v1/UpdateService/FirmwareInventory/<item> | jq '{Name, Version, Updateable}'
# Push an update package (multipart, where supported)
curl -s -u "$BMC_USER:$BMC_PASS" --cacert bmc-ca.pem -X POST \
"https://$BMC$(curl -s -u "$BMC_USER:$BMC_PASS" --cacert bmc-ca.pem https://$BMC/redfish/v1/UpdateService | jq -r .MultipartHttpPushUri)" \
-F 'UpdateParameters={"Targets":[]};type=application/json' -F 'UpdateFile=@firmware.fwpkg'
# UpdateParameters is a required part ({} or "Targets":[] if you have nothing to set).
# Targets [] means "the service chooses"; on DGX H100 name the component, e.g.
# {"Targets":["/redfish/v1/UpdateService/FirmwareInventory/HostBMC_0"]}
# HTTP 202 Accepted. Location: the task monitor URI (e.g. /redfish/v1/TaskService/TaskMonitors/5);
# the body is the Task (/redfish/v1/TaskService/Tasks/5).
# Poll the task until it completes
$R https://$BMC/redfish/v1/TaskService/Tasks/5 | jq '{TaskState, PercentComplete, Messages}'
# Pull variant: the BMC downloads the image itself (where UpdateService lists SimpleUpdate)
$R -X POST https://$BMC/redfish/v1/UpdateService/Actions/UpdateService.SimpleUpdate \
-d '{"ImageURI":"https://repo.example.internal/fw/package.fwpkg","TransferProtocol":"HTTPS"}'
# Push (MultipartHttpPushUri): you upload the file. Pull (SimpleUpdate): the BMC fetches a URL.
TaskState is Completed (or Exception). Many components only activate the new firmware after a host reboot, a power cycle, or a BMC reset. IPMI has no standard firmware-update mechanism; this is one of Redfish's big wins.nvfwupd utility, which drives the BMC's Redfish update service) plus release-specific procedures and ordering. Follow the DGX firmware release notes for your model rather than flashing components ad hoc.12. Events, logs, and telemetry
| Mechanism | How | Use |
|---|---|---|
| Polling logs | GET LogServices/.../Entries (Redfish) or sel elist (IPMI) | Post-mortem: "what did the hardware report before the crash?" |
| Event subscriptions (push) | POST to /redfish/v1/EventService/Subscriptions with a Destination URL | Alerts sent to your collector when something happens |
| Server-Sent Events | A long-lived HTTPS stream (ServerSentEventUri) | Live event feed without running a listener |
| SNMP traps / PET | Legacy alerting (IPMI Platform Event Traps) | Older monitoring stacks |
| Telemetry | TelemetryService metric reports, or polling Sensors | Power and thermal trends at fleet scale |
$R -X POST https://$BMC/redfish/v1/EventService/Subscriptions -d '{
"Destination": "https://monitor.example.internal/redfish-events",
"Protocol": "Redfish",
"EventFormatType": "Event"
}'
dmesg), and GPU health comes from DCGM / nvsm.13. CLI tools and libraries
| Tool | What it is | Example |
|---|---|---|
| ipmitool | The standard IPMI CLI (local or LAN) | ipmitool -I lanplus -H $BMC -U $U -E sel elist |
| curl + jq | Raw Redfish over HTTPS; universal | See Common tasks with curl |
| redfishtool | DMTF's Python CLI that wraps common Redfish operations | redfishtool -r $BMC -u $U -p $P Systemsredfishtool -r $BMC -u $U -p $P Systems reset GracefulRestart |
| DMTF Redfish-Tools / redfish-utilities | Scripts such as rf_power_reset.py, rf_sensor_list.py, rf_logs.py, rf_update.py | rf_sensor_list.py -u $U -p $P -r $BMC |
| python-redfish-library | DMTF Python client library for your own scripts | redfish.redfish_client(base_url=..., username=..., password=...) |
| Ansible | community.general.redfish_info / redfish_command / redfish_config | Fleet-wide power, boot, and inventory automation |
| Redfish Service Validator | DMTF tool that checks a BMC's Redfish implementation against the schemas | Compliance testing |
| Redfish Mockup Server | Serves a recorded Redfish tree locally | Developing scripts without real hardware |
| BCM (cmsh / Base View) | Uses the BMC (IPMI or Redfish) for node power control | cmsh -c "device; power status -n dgx01" |
--help.14. IPMI ↔ Redfish mapping
| Task | ipmitool | Redfish |
|---|---|---|
| Power state | chassis power status | GET Systems/{id} → PowerState |
| Power cycle | chassis power cycle | POST ComputerSystem.Reset {"ResetType":"PowerCycle"} |
| Graceful shutdown | chassis power soft | GracefulShutdown |
| PXE next boot | chassis bootdev pxe | PATCH Systems/{id} Boot.BootSourceOverrideTarget=Pxe, Enabled=Once |
| Event log | sel elist | GET LogServices/.../Entries |
| Sensors | sdr elist / sensor | GET Chassis/{id}/Thermal, Power, or Sensors |
| Inventory | fru print | GET Chassis / Systems (SerialNumber, PartNumber, Model), Processors, Memory |
| BMC info / reset | mc info / mc reset cold | GET Managers/{id} / POST Manager.Reset |
| BMC network | lan print 1 | GET Managers/{id}/EthernetInterfaces |
| Users | user list 1 | GET AccountService/Accounts |
| Console | sol activate | Serial console / SSH to the BMC / web KVM (vendor-specific) |
| Firmware update | (not standardized) | UpdateService + TaskService |
| Event push | SNMP / PET traps | EventService subscriptions, SSE |
15. DGX, HGX, and BCM specifics
- DGX BMC port: a dedicated RJ-45 management port that connects to the cluster's OOB management network, separate from the in-band management and compute fabrics.
- First-boot credentials: DGX systems require setting or changing the BMC password. Don't leave factory defaults on a reachable network.
- Remote KVM and virtual media: from the BMC web UI you can watch the console during boot and mount an ISO, for example to reinstall DGX OS.
- HGX baseboard management: on HGX/DGX systems, the GPU baseboard has its own management controller (the HMC, HGX Management Controller) that handles GPU and NVSwitch firmware and telemetry. It's typically reached through the BMC's Redfish service, which is why GPU-tray firmware appears in the Redfish firmware inventory. See the HGX firmware guide.
- In-band vs out-of-band on DGX:
nvsmand DCGM run in the host OS (in-band). The BMC/Redfish works even when the OS is down.
BCM and the BMC
- BCM stores each node's BMC address and credentials (BMC settings at the cluster, category, or node level) and uses them for power control through IPMI or Redfish.
- That's how BCM can power-cycle a hung node or force a PXE reprovision without anyone touching the server.
cmsh
% device
% power status -n dgx01..dgx04
% power reset -n dgx03 # hard reset through the BMC
% power on -c dgx-h100 # a whole category
cmsh help (for example under partition / category / device BMC settings) for your release.power reset or Redfish PowerCycle) → watch the boot on the KVM console → once the OS is up, use in-band tools (nvsm, dcgmi diag, dmesg for Xids). See DGX/BCM node recovery and NVSM mastery.DGX H100/H200 Redfish recipes
These follow NVIDIA's DGX H100/H200 user guide ("Redfish APIs Support"). The DGX B200 and B300 guides document the same GPU power path; check your model's guide for the rest. $R is the curl alias from section 10.
| Resource | DGX H100/H200 ID |
|---|---|
| Host system | /redfish/v1/Systems/DGX (pending "future" settings: Systems/DGX/SD) |
| Chassis (platform sensors) | /redfish/v1/Chassis/DGX |
| The BMC | /redfish/v1/Managers/BMC |
| GPU tray, through the HMC | /redfish/v1/Systems/HGX_Baseboard_0; GPUs are Processors/GPU_SXM_1 to GPU_SXM_8 |
| Firmware targets | HGX_0 (the whole GPU tray), HostBMC_0, HostBIOS_0, EROT_BMC_0, EROT_BIOS_0, CPLDMB_0 (CPU tray CPLD), CPLDMID_0 (midplane CPLD), PSU_0 to PSU_5, PCIeSwitch_0/1, PCIeRetimer_0/1 |
# Reach the BMC from the host itself: the BMC sits at 169.254.0.17 on an internal link.
# Give the host's interface on that link (an enx... USB NIC) an address in 169.254.0.0/16:
sudo ip addr add 169.254.0.18/16 dev enx<...> # NVIDIA's example uses ifconfig
nvfwupd -t ip=169.254.0.17 username=$BMC_USER password=$BMC_PASS show_version
# Set the BMC password after first boot: 13-20 characters, upper + lower + digit + special, no spaces
$R -X PATCH -H 'If-Match: *' https://$BMC/redfish/v1/AccountService/Accounts/2 -d '{"Password":"'"$NEW_BMC_PASS"'"}'
# Reboot the BMC / factory-reset the BMC configuration
$R -X POST https://$BMC/redfish/v1/Managers/BMC/Actions/Manager.Reset -d '{"ResetType":"ForceRestart"}'
$R -X POST https://$BMC/redfish/v1/Managers/BMC/Actions/Manager.ResetToDefaults -d '{"ResetType":"ResetAll"}'
# Firmware: push to UpdateService/upload; parameters.json picks the target
echo '{"Targets":["/redfish/v1/UpdateService/FirmwareInventory/HGX_0"]}' > parameters.json
curl -s -u "$BMC_USER:$BMC_PASS" --cacert bmc-ca.pem -H 'Expect:' -X POST https://$BMC/redfish/v1/UpdateService/upload \
-F 'UpdateParameters=@parameters.json;type=application/json' -F 'UpdateFile=@nvfw_HGX_DGXH100-H200x8_<ver>.fwpkg'
# GPU tray: always target HGX_0 (nvfw_HGX_... package).
# CPU/motherboard tray: name the component, e.g. HostBMC_0 or CPLDMB_0 (nvfw_DGX_... package).
# A package that isn't newer is skipped unless ForceUpdate is set (204 on success, 400 if already set):
$R -X PATCH -H 'If-Match: *' https://$BMC/redfish/v1/UpdateService -d '{"HttpPushUriOptions":{"ForceUpdate":true}}'
# After the new image has booted, copy it to the other flash bank ("background copy"):
$R -X POST https://$BMC/redfish/v1/UpdateService/Actions/Oem/NvidiaUpdateService.CommitImage \
-d '{"Targets":["/redfish/v1/UpdateService/FirmwareInventory/HostBIOS_0"]}'
# BIOS settings: find the attribute registry, read current values, PATCH the future settings (SD)
$R https://$BMC/redfish/v1/Registries | jq -r '.Members[]."@odata.id"' | grep BiosAttributeRegistry
$R https://$BMC/redfish/v1/Systems/DGX/Bios | jq .Attributes
$R -X PATCH -H 'If-Match: *' https://$BMC/redfish/v1/Systems/DGX/Bios/SD -d '{"Attributes":{"<AttrName>":"<Value>"}}'
# Every BIOS attribute change needs a power cycle. A BIOS update afterwards needs another one.
$R -X POST https://$BMC/redfish/v1/UpdateService/Actions/Oem/NvidiaUpdateService.ClearNVRAM \
-d '{"Targets":["/redfish/v1/UpdateService/FirmwareInventory/HostBIOS_0"]}' # clear BIOS settings to defaults
# Persistent boot order (unlike BootSourceOverride "Once"): read it, decode the entries, PATCH SD
$R https://$BMC/redfish/v1/Systems/DGX/SD | jq .Boot.BootOrder # ["Boot0000","Boot000F","Boot0004",...]
$R https://$BMC/redfish/v1/Systems/DGX/BootOptions/0004 | jq '{DisplayName, UefiDevicePath}' # MAC is in the path
$R -X PATCH https://$BMC/redfish/v1/Systems/DGX/SD -H 'If-Match: "<etag from the GET>"' \
-d '{"Boot":{"BootOrder":["Boot0004","Boot0000","Boot000F"]}}'
# Secure Boot state (replacing the Platform Key: disable, DELETE the PK cert, POST a new PEM, reboot)
$R https://$BMC/redfish/v1/Systems/DGX/SecureBoot | jq '{SecureBootEnable, SecureBootMode}'
# Virtual media: mount a DGX OS ISO over NFS (reinstall with nobody on site)
$R -X POST https://$BMC/redfish/v1/Managers/BMC/Actions/Oem/AMIVirtualMedia.EnableRMedia -d '{"RMediaState":"Enable"}'
$R https://$BMC/redfish/v1/Managers/BMC/VirtualMedia | jq -r '.Members[]."@odata.id"' # CD1 .. CD4
$R -X POST https://$BMC/redfish/v1/Managers/BMC/VirtualMedia/CD1/Actions/VirtualMedia.InsertMedia \
-d '{"Image":"//nfs.example.internal/isos/DGXOS-7.x.iso","TransferProtocolType":"NFS"}' # returns a Task
$R https://$BMC/redfish/v1/Managers/BMC/VirtualMedia/CD1 | jq '{ImageName, Inserted, TransferProtocolType}'
# Then put "UEFI: AMI Virtual CDROM0 1.00" first in the boot order and power-cycle.
# Unmount afterwards: POST .../VirtualMedia/CD1/Actions/VirtualMedia.EjectMedia
# Sensors, GPU power, telemetry. DGX collections return 75 members per page.
$R https://$BMC/redfish/v1/Chassis/DGX/Sensors | jq -r '."Members@odata.nextLink"' # .../Sensors?$skip=75
$R https://$BMC/redfish/v1/Managers/BMC/LogServices/SEL/Entries | jq '.Members | length' # also paged at 75
$R https://$BMC/redfish/v1/Systems/HGX_Baseboard_0/Processors/GPU_SXM_1/EnvironmentMetrics | jq .PowerLimitWatts
# Reading = current GPU power (W), SetPoint = current power limit, AllowableMin/AllowableMax = range
$R https://$BMC/redfish/v1/TelemetryService/MetricReportDefinitions/HGX_PlatformEnvironmentMetrics_0
# BMC debug bundle for NVIDIA support
$R -X POST https://$BMC/redfish/v1/Managers/BMC/LogServices/DiagnosticLog/Actions/LogService.CollectDiagnosticData \
-d '{"DiagnosticDataType":"OEM","OEMDiagnosticDataType":"ALL"}' # returns a Task
$R https://$BMC/redfish/v1/TaskService/Tasks/<n> | jq '.TaskState, .Messages[].Message' # names the Attachment
curl -s -u "$BMC_USER:$BMC_PASS" --cacert bmc-ca.pem -o nvidiadiag.tar.gz \
https://$BMC/redfish/v1/Managers/BMC/LogServices/DiagnosticLog/Attachment/<file>.tar.gz
# BMC configuration backup and restore (upload an AES key first)
# Managers/BMC/Actions/Oem/NvidiaManager.UploadAESKey, then .BackupConfig / .RestoreConfig
# Power capping: Managers/BMC/NodeManager (Domains, Policies)
DGX (system and chassis) and BMC (manager). The GPU tray is its own system, HGX_Baseboard_0, reached through the BMC. BIOS changes go to Bios/SD (future settings) and need a power cycle. To update the GPU tray, target HGX_0.Oem (AMIVirtualMedia, NvidiaManager, NvidiaUpdateService) are vendor extensions, not DMTF standard, so other BMCs won't have them. The HMC rejects ForceRestart, GracefulRestart, and GracefulShutdown resets for security reasons.16. Command builder
17. "Which tool?" chooser
18. Sample outputs
What the commands in this guide print. Expand a command to see a typical result and what to check in it.
ipmitool -I lanplus -H dgx01-bmc -U admin -E chassis power statusHost power state via the BMC
Chassis Power is onWhat to look at: On with the OS not answering points to a hung OS, not a dead box; check the SEL next. An error like Unable to establish IPMI v2 / RMCP+ session means a wrong password, a missing -I lanplus, or an unreachable BMC.
ipmitool -I lanplus -H dgx01-bmc -U admin -E chassis bootdev pxe options=persistent,efibootSet PXE boot (persistent, UEFI)
Set Boot Device to pxeWhat to look at: The reply only confirms the request. Without options=persistent it applies to the next boot only, and without efiboot a UEFI-only DGX may try a legacy PXE boot and fall through to disk.
ipmitool -I lanplus -H dgx01-bmc -U admin -E sel elistSystem Event Log with decoded timestamps
1 | 11/02/2026 | 08:12:44 | Event Logging Disabled #0x07 | Log area reset/cleared | Asserted
2 | 11/02/2026 | 08:15:03 | Power Unit #0x01 | Power off/down | Asserted
3 | 11/02/2026 | 08:16:40 | System Event #0x52 | Timestamp Clock Sync | Asserted
4 | 11/08/2026 | 02:14:51 | Power Supply PSU1_Status | Failure detected | Asserted
5 | 11/08/2026 | 02:14:52 | Power Unit PSU_Redundancy | Redundancy Lost | Asserted
6 | 11/08/2026 | 23:41:07 | OS Critical Stop #0x46 | Run-time critical stop | AssertedWhat to look at: Read the sensor and event columns with their timestamps: Failure detected on a PSU, Redundancy Lost, or OS Critical Stop (a kernel panic) are the entries that explain an outage. Save the log with sel save before any sel clear.
ipmitool -I lanplus -H dgx01-bmc -U admin -E sdr elistAll sensors with status and reading
Inlet_Temp | 30h | ok | 64.1 | 23 degrees C
CPU0_Temp | 31h | ok | 3.0 | 51 degrees C
CPU1_Temp | 32h | ok | 3.1 | 49 degrees C
GPU_SXM_1_Temp | 40h | ok | 11.1 | 38 degrees C
...
FAN_1_F | 60h | ok | 29.1 | 7800 RPM
FAN_2_F | 61h | ok | 29.2 | 7840 RPM
...
PSU0_Status | 80h | ok | 10.1 | Presence detected
PSU1_Status | 81h | cr | 10.2 | Presence detected, Failure detected
PSU0_Input_Power | 88h | ok | 10.1 | 1020 Watts
...What to look at: The third column is the status: ok is normal, while cr (critical), nc (non-critical) or nr (non-recoverable) need attention. Here PSU1 shows Failure detected, matching the SEL entry.
ipmitool -I lanplus -H dgx01-bmc -U admin -E fru printSerial and part numbers (FRU)
FRU Device Description : Builtin FRU Device (ID 0)
Chassis Type : Rack Mount Chassis
Chassis Part Number : 965-24387-0002-003
Chassis Serial : 1660423000018
Board Mfg Date : Mon Mar 4 09:00:00 2024 UTC
Board Mfg : NVIDIA
Board Product : DGX H100
Board Serial : 1572423000101
Board Part Number : 699-24387-0001-000
Product Manufacturer : NVIDIA
Product Name : DGX H100
Product Part Number : 920-24387-2540-000
Product Serial : 1660423000018
FRU Device Description : PSU0 (ID 4)
Product Manufacturer : (example PSU vendor)
Product Name : 3300W CRPS
Product Serial : PS0-dgx01-4411
FRU Device Description : PSU1 (ID 5)
...What to look at: Use the Product Serial and part numbers when opening a support case or ordering a replacement part. Blank or Unknown fields usually mean a FRU EEPROM that was not programmed after a board swap.
ipmitool -I lanplus -H dgx01-bmc -U admin -E mc infoBMC identity and firmware version
Device ID : 32
Device Revision : 1
Firmware Revision : 25.03
IPMI Version : 2.0
Manufacturer ID : 5703
Manufacturer Name : NVIDIA
Product ID : 5136 (0x1410)
Product Name : Unknown (0x1410)
Device Available : yes
Provides Device SDRs : yes
Additional Device Support :
Sensor Device
SDR Repository Device
SEL Device
FRU Inventory Device
IPMB Event Receiver
IPMB Event Generator
Chassis Device
Aux Firmware Rev Info :
0x01
0x00
0x00
0x00What to look at: Firmware Revision is the BMC's own firmware (major.minor), not the BIOS. After a BMC update and mc reset cold it should show the new version; Device Available: no means the BMC is still booting.
ipmitool -I lanplus -H dgx01-bmc -U admin -E lan print 1BMC network settings on channel 1
Set in Progress : Set Complete
Auth Type Support : MD5 PASSWORD
Auth Type Enable : Callback : MD5 PASSWORD
: User : MD5 PASSWORD
: Operator : MD5 PASSWORD
: Admin : MD5 PASSWORD
: OEM :
IP Address Source : Static Address
IP Address : 10.0.100.11
Subnet Mask : 255.255.255.0
MAC Address : 7c:c2:55:01:4a:1e
SNMP Community String : AMI
IP Header : TTL=0x40 Flags=0x40 Precedence=0x00 TOS=0x10
...
Default Gateway IP : 10.0.100.1
Default Gateway MAC : 00:00:5e:00:01:01
802.1q VLAN ID : Disabled
802.1q VLAN Priority : 0
RMCP+ Cipher Suites : 0,1,2,3,6,7,8,11,12,15,16,17
Cipher Suite Priv Max : XaaaXXaaaXXaaXX
: X=Cipher Suite Unused
: c=CALLBACK
: u=USER
: o=OPERATOR
: a=ADMIN
: O=OEM
Bad Password Threshold : 0What to look at: Check IP Address Source, IP Address and gateway: DHCP Address with 0.0.0.0 means the BMC never got a lease. In Cipher Suite Priv Max the first character is suite 0, so it must be X; an a there means anyone can log in without a valid password.
ipmitool -I lanplus -C 0 -H dgx05-bmc -U admin -P wrong chassis power statusProve cipher suite 0 is disabled
Error in open session response message : no cipher suite match with proposed security algorithms
Error: Unable to establish IPMI v2 / RMCP+ sessionWhat to look at: This failure is the result you want after disabling suite 0. If it instead prints Chassis Power is on with a wrong password, suite 0 is still open and the BMC is wide open on the network.
ipmitool -I lanplus -H dgx01-bmc -U admin -E sol activateSerial-over-LAN host console
[SOL Session operational. Use ~? for help]
Ubuntu 22.04.4 LTS dgx01 ttyS0
dgx01 login:What to look at: SOL Session operational means you are attached to the host's serial console; a login prompt or kernel panic text tells you what the OS is doing. SOL payload already active on another session means someone else is attached; exit with ~. when done.
curl -si --cacert bmc-ca.pem -X POST https://$BMC/redfish/v1/SessionService/Sessions -H 'Content-Type: application/json' -d '{"UserName":"'"$BMC_USER"'","Password":"'"$BMC_PASS"'"}'Redfish session login
HTTP/1.1 201 Created
Location: /redfish/v1/SessionService/Sessions/17
X-Auth-Token: token-redacted
OData-Version: 4.0
Content-Type: application/json; charset=utf-8
Cache-Control: no-store
{
"@odata.id": "/redfish/v1/SessionService/Sessions/17",
"@odata.type": "#Session.v1_5_0.Session",
"Id": "17",
"Name": "User Session",
"UserName": "admin",
"ClientOriginIPAddress": "10.0.100.254"
}What to look at: Keep the X-Auth-Token header value for later requests and the Location for logging out with DELETE. 401 means bad credentials, and 503 with SessionLimitExceeded means leaked sessions are filling the BMC's slots.
$R https://$BMC/redfish/v1/Systems | jq -r '.Members[]."@odata.id"'Discover the system ID
/redfish/v1/Systems/DGXWhat to look at: On DGX H100 the host system is Systems/DGX, and the GPU baseboard appears separately as Systems/HGX_Baseboard_0. Use what the collection returns rather than guessing IDs; a 404 on a guessed path is the usual result.
$R https://$BMC/redfish/v1/Systems/DGX | jq '{PowerState, Status}'Host power state and health
{
"PowerState": "On",
"Status": {
"State": "Enabled",
"Health": "OK"
}
}What to look at: PowerState On with Health OK is normal. Health Warning or Critical means look at the sensors and log entries, and State StandbyOffline means the host is powered off.
$R https://$BMC/redfish/v1/Systems/DGX | jq '.Actions."#ComputerSystem.Reset"'Allowed reset types for this host
{
"target": "/redfish/v1/Systems/DGX/Actions/ComputerSystem.Reset",
"ResetType@Redfish.AllowableValues": [
"On",
"ForceOff",
"GracefulShutdown",
"GracefulRestart",
"ForceRestart",
"PowerCycle",
"Nmi"
]
}What to look at: POST to the target URI with one of these exact ResetType strings. A value not in the list, such as Reset or Reboot, is rejected with a 400 PropertyValueNotInList error.
$R https://$BMC/redfish/v1/Chassis/DGX/Thermal | jq '.Temperatures[] | {Name, ReadingCelsius, Status}'Chassis temperature readings
{
"Name": "Inlet_Temp",
"ReadingCelsius": 23,
"Status": {
"Health": "OK"
}
}
{
"Name": "CPU0_Temp",
"ReadingCelsius": 51,
"Status": {
"Health": "OK"
}
}
...What to look at: Look at ReadingCelsius together with Status.Health. A high inlet temperature points to the data center cooling, and Health Warning or Critical on one component points to that part or its fans.
$R https://$BMC/redfish/v1/Managers/BMC/LogServices/SEL/Entries | jq '.Members[] | {Created, Severity, Message}'SEL entries via Redfish
{
"Created": "2026-11-08T02:14:51+00:00",
"Severity": "Critical",
"Message": "Power Supply: Failure detected"
}
{
"Created": "2026-11-08T02:14:52+00:00",
"Severity": "Critical",
"Message": "Power Unit: Redundancy Lost"
}
{
"Created": "2026-11-08T23:41:07+00:00",
"Severity": "Critical",
"Message": "OS Critical Stop: Run-time critical stop"
}
...What to look at: This is the same SEL as ipmitool sel elist, with ISO timestamps and a Severity field. Filter on Severity Critical; if the times do not line up with host logs, check the BMC NTP settings.
$R https://$BMC/redfish/v1/Systems/DGX | jq '{Model, SerialNumber, BiosVersion, ProcessorSummary, MemorySummary}'Host inventory summary
{
"Model": "DGX H100",
"SerialNumber": "1660423000018",
"BiosVersion": "01.05.04",
"ProcessorSummary": {
"Count": 2,
"Model": "Intel(R) Xeon(R) Platinum 8480C"
},
"MemorySummary": {
"TotalSystemMemoryGiB": 2048
}
}What to look at: A DGX H100 should show 2 CPUs and 2048 GiB of memory. A lower TotalSystemMemoryGiB means a DIMM failed or was disabled, and BiosVersion confirms a BIOS update actually took effect.
$R https://$BMC/redfish/v1/Chassis/DGX/Sensors | jq '."Members@odata.count", ."Members@odata.nextLink"'Sensor count and paging link
212
"/redfish/v1/Chassis/DGX/Sensors?$skip=75"What to look at: Members@odata.count is the total, but each page holds only 75 members on DGX. A nextLink that is not null means more pages to fetch with $skip; a script that ignores it silently misses most sensors.
$R https://$BMC/redfish/v1/TaskService/Tasks/5 | jq '{TaskState, PercentComplete, Messages}'Poll a firmware update task
{
"TaskState": "Running",
"PercentComplete": 45,
"Messages": [
{
"@odata.type": "#Message.v1_1_1.Message",
"Message": "The task with Id '5' has started.",
"MessageArgs": [
"5"
],
"MessageId": "TaskEvent.1.0.TaskStarted",
"Resolution": "None.",
"Severity": "OK"
}
]
}What to look at: Poll until TaskState is Completed and PercentComplete is 100. Exception, Killed, or a message with Severity Critical means the update failed, so do not power-cycle until you have read why.
$R https://$BMC/redfish/v1/Systems/DGX/SD | jq .Boot.BootOrderPersistent UEFI boot order (DGX H100)
[
"Boot0000",
"Boot000F",
"Boot0004",
"Boot0001",
"Boot0002"
]What to look at: The first entry boots first; decode each one through Systems/DGX/BootOptions/<n> to see its DisplayName and MAC. For PXE reprovisioning the right network BootOption must be first, and the PATCH needs the ETag from this GET.
$R https://$BMC/redfish/v1/Systems/HGX_Baseboard_0/Processors/GPU_SXM_1/EnvironmentMetrics | jq .PowerLimitWattsGPU power reading and limit (DGX H100)
{
"AllowableMax": 700,
"AllowableMin": 200,
"DefaultSetPoint": 700,
"Reading": 312.47,
"SetPoint": 700
}What to look at: Reading is the GPU's current power draw in watts and SetPoint is its active limit, within AllowableMin to AllowableMax. A SetPoint well below 700 W on an H100 SXM explains lower-than-expected performance.
19. Hands-on lab: simulated tickets
Five tickets from a simulated BCM head node, head01, which sits on the OOB management network with five DGX H100 nodes (dgx01–dgx05, BMCs dgx01-bmc–dgx05-bmc):
- OOB-101: a hung node. Read the SEL, keep the evidence, pick a power action that actually works on a hung OS, then find the GPU error in the host's log.
- RF-202: PXE-boot a node once with raw Redfish calls: service root, sessions, discovering IDs, ETags and
If-Match, the Reset action, and logging out. - FW-303: a BMC firmware update through
UpdateServiceandTaskServiceon a BMC whose session slots are full, without touching the host's running job. - NET-404: a new node's BMC has no usable address and allows IPMI cipher suite 0. Fix it in-band from the host, then prove BCM can power it and suite 0 is refused.
- HW-505: a failed PSU on a node that's running jobs. Collect the RMA evidence with
ipmitool(sdr,sel,fru print,mc info), find the one fact IPMI can't give you, and light the locator LED, all without a power action.
$BMC_USER) expand, | jq supports common paths and {A, B} projections, $(...) doesn't run. Up/Down recall history; Tab completes hosts, files, tokens, and ETags.Answer key for this ticket (spoilers)
One clean path; other orders work too. The checklist tracks results, not exact commands.
20. Scenarios
21. Searchable reference
| Item | Meaning |
|---|
22. Flashcards (click to flip)
23. Exam traps in one list
- Redfish = DMTF, REST, HTTPS (443), JSON. IPMI = binary, RMCP+ over UDP 623.
- Redfish's entry point is
/redfish/v1/. Discover IDs fromMembers; don't guess them. - Systems = power and boot of the host. Chassis = sensors, fans, PSUs. Managers = the BMC.
- Power actions are POSTs to an Action (
ComputerSystem.Reset), not PATCHes ofPowerState. - PXE once = PATCH
BootwithBootSourceOverrideTarget: Pxe,BootSourceOverrideEnabled: Once. - 202 Accepted = async; poll the task. 412 = stale ETag.
- Session token header =
X-Auth-Token. DELETE sessions when done. ipmitool -I lanplus= IPMI 2.0.-Ereads the password fromIPMI_PASSWORD.mc reset coldresets the BMC.chassis power resetresets the server.chassis power soft= graceful;off= hard.- The BMC works with the host OS down.
nvsmand DCGM need the OS. - Xid errors are in the host kernel log, not the BMC SEL.
- IPMI weaknesses: cipher suite 0, RAKP hash disclosure → isolate BMC networks, prefer Redfish.
- Firmware updates: Redfish
UpdateService(and NVIDIA's tooling on DGX). IPMI has no standard way. - Push = upload to
MultipartHttpPushUri(DGX:UpdateService/upload). Pull =SimpleUpdatewith anImageURI. DGX GPU tray target =HGX_0; same or older versions needForceUpdate. - DGX H100/H200 IDs:
Systems/DGX,Chassis/DGX,Managers/BMC; GPUs underSystems/HGX_Baseboard_0/Processors/GPU_SXM_n. - BIOS changes are PATCHed to the settings resource (
@Redfish.Settings; DGX:Bios/SD), notBiositself, and apply after a power cycle. BootSourceOverrideEnabled: Once= next boot only.Boot.BootOrder= persistent order.- Big collections are paged: follow
Members@odata.nextLink/$skip(DGX: 75 per page). - Remote ISO =
VirtualMedia.InsertMediaunderManagers/{id}/VirtualMedia(DGX: enable it first withAMIVirtualMedia.EnableRMedia). - Identify LED =
LocationIndicatorActiveon the Chassis. GPU power limit out-of-band =EnvironmentMetrics→PowerLimitWatts.SetPoint.