Commit 501527d ("[http] Treat any unexpected connection close as an
error") introduced a regression causing HTTP SAN booting to fail. At
the end of the response to the HEAD request, the call to http_done()
would erroneously believe that the server had disconnected in the
middle of the HTTP headers.
Fix by treating the header block from a HEAD request as a trailer
block. This fixes the problem and also simplifies the logic in
http_rx_header().
Reported-by: Shao Miller <shao.miller@yrdsb.edu.on.ca>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The FTP SIZE command allows us to get the size of a particular file,
as a consequence, we can now show proper transfer progression while
fetching a file using the FTP protocol.
Signed-off-by: Marin Hannache <git@mareo.fr>
Modified-by: Michael Brown <mcb30@ipxe.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
iPXE currently checks that the server has not closed the connection
mid-stream (i.e. in the middle of a chunked transfer, or before the
specified Content-Length has been received), but does not check that
the server got as far as starting to send data. Consequently, if the
server closes the connection before any data is transferred (e.g. if
the server gives up waiting while iPXE performs the validation steps
for TLS), then iPXE will treat this as a successful transfer of a
zero-length file.
Fix by checking the RX connection state, and forcing an error if the
server has closed the connection at an unexpected point.
Originally-fixed-by: Marin Hannache <mareo@mareo.fr>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Whenever memory pressure causes a queued packet to be discarded (and
so retransmitted), reduce the maximum TCP window to a size that would
have prevented the discard.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Discarding the active ARP cache entry in the middle of a download will
substantially disrupt the TCP stream. Try to minimise any such
disruption by treating ARP cache entries as expensive, and discarding
them only when nothing else is available to discard.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The current logic is to process at most one received packet per call
to net_poll(), on the basis that refilling the hardware descriptor
ring should be delayed as little as possible. However, this limits
the rate at which packets can be processed and ultimately ends up
adding latency which, in turn, limits the achievable throughput.
With temporary modifications in place to essentially remove all
resource constraints (heap size increased to 16MB, RX descriptor ring
increased to 64 descriptors) and a TCP window size of 1MB, the
throughput on a gigabit (i.e. 119MBps) network can be observed to fall
off exponentially from around 115MBps to around 75MBps. Changing
net_poll() to process all received packets results in a steady
119MBps throughput.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Each ARP cache entry maintains a transmission queue, which is sent out
as soon as the link-layer address is known. If multiple packets are
queued, then it is possible for memory pressure to cause the ARP cache
discarder to be invoked during transmission of the first packet, which
may cause the ARP cache entry to be deleted before the second packet
can be sent. This results in an invalid pointer dereference.
Avoid this problem by reference-counting ARP cache entries and
ensuring that an extra reference is held while processing the
transmission queue, and by using list_first_entry() rather than
list_for_each_entry_safe() to traverse the queue.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Commit ea61075 ("[tcp] Add support for TCP window scaling") introduced
a potential NULL pointer dereference by referring to the connection's
send window scale before checking whether or not the connection is
known.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The default maximum plaintext fragment length for TLS is 16kB, which
is a substantial amount of memory for iPXE to have to allocate for a
temporary decryption buffer.
Reduce the memory footprint of TLS connections by requesting a maximum
fragment length of 2kB.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The maximum unscaled TCP window (64kB) implies a maximum bandwidth of
around 300kB/s on a WAN link with an RTT of 200ms. Add support for
the TCP window scaling option to remove this upper limit.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Calculating the TCP/IP checksum on received packets accounts for a
substantial fraction of the response latency.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
FCoE requires the use of multiple local unicast link-layer addresses.
To avoid the complexity of managing multiple addresses, iPXE operates
in promiscuous mode. As a consequence, any unicast packets with
non-matching IPv4 addresses are rejected at the IPv4 layer (rather
than at the link layer).
This can cause problems when issuing a second DHCP request: if the
address chosen by the DHCP server does not match the existing address,
then the DHCP response will itself be rejected.
Fix by requesting a broadcast response from the DHCP server if the
network interface already has any IPv4 addresses.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Provide HTTP Basic authentication credentials only in response to a
401 Unauthorized response from the server.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Some headers can modify the meaning of the response code. For
example, a WWW-Authenticate header can change the interpretation of a
401 Unauthorized response from "Access denied" to "Please
authenticate".
Signed-off-by: Michael Brown <mcb30@ipxe.org>
iSCSI generally includes a full SCSI response only when an error
occurs. iscsi_scsi_done() currently passes the NULL response through
to scsi_response(), which ends up causing scsicmd_response() to
dereference a NULL pointer.
Fix by calling scsi_response() only if we have a non-NULL response.
Reported-by: Brendon Walsh <brendonwalsh@niamu.com>
Tested-by: Brendon Walsh <brendonwalsh@niamu.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
X.509 certificate processing currently produces an overwhelming amount
of debugging information. Move some of this from DBGLVL_LOG to
DBGLVL_EXTRA, to make the output more manageable.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Automatically attempt to download any required cross-signing
certificates from http://ca.ipxe.org/auto, in order to enable the use
of standard SSL certificates issued by public CAs.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
To allow for automatic download of cross-signing certificates and for
OCSP, the validation of certificates must be an asynchronous process.
Create a stub validator which uses a job-control interface to report
the result of certificate validation.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
At present, certificate chain validation is treated as an
instantaneous process that can be carried out using only data that is
already in memory. This model does not allow for validation to
include non-instantaneous steps, such as downloading a cross-signing
certificate, or determining certificate revocation status via OCSP.
Redesign the internal representation of certificate chains to allow
chains to outlive the scope of the original source of certificates
(such as a TLS Certificate record).
Allow for certificates to be cached, so that each certificate needs to
be validated only once.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
http_step() allocates a potentially large block of storage (since the
URI can be arbitrarily long), and can be invoked as part of an already
deep call stack via xfer_window_changed().
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Allow TFTP to be configured out by moving the next-server setting
definition (which is used by autoboot.c) from tftp.c to settings.c.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
sizeof(cipherspec) is obviously wrong in this context, because it will
only zero the first 4 or 8 bytes (cipherspec is a pointer).
This problem was reported by cppcheck.
Signed-off-by: Stefan Weil <sw@weilnetz.de>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Use a private ANSI escape sequence to convey the priority of an
internal syslog() message through to the syslog server.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
An ANSI escape sequence context cannot be shared between multiple
users. Make the ANSI escape sequence context part of the line console
definition and provide individual contexts for each user.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The output from text-based user interfaces such as the "config"
command is not generally meaningful for logfile-based consoles such as
syslog and vmconsole.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Add the concept of a "console usage", such as "standard output" or
"debug messages". Allow usages to be associated with each console
independently. For example, to send debugging output via the serial
port, while preventing it from appearing on the local console:
#define CONSOLE_SERIAL CONSOLE_USAGE_ALL
#define CONSOLE_PCBIOS ( CONSOLE_USAGE_ALL & ~CONSOLE_USAGE_DEBUG )
If no usages are explicitly specified, then a default set of usages
will be applied. For example:
#define CONSOLE_SERIAL
will have the same affect as
#define CONSOLE_SERIAL CONSOLE_USAGE_ALL
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Simplify code by recording the active handshake digest algorithm as a
session parameter. (Note that we must still accumulate digests for
all supported algorithms, since we don't know which digest will
eventually be used until we receive the Server Hello.)
Signed-off-by: Michael Brown <mcb30@ipxe.org>
TLSv1.1 and earlier use a hybrid of MD5 and SHA-1 to generate digests
over the handshake messages. Formalise this as a separate digest
algorithm "md5+sha1".
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Validate the server certificate against the trusted root certificate
store. The server must provide a complete certificate chain, up to
and including the trusted root certificate that is embedded into iPXE.
Note that the date and time are not yet validated.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
All axTLS files are now vanilla versions of the upstream axTLS files,
with one minor exception: the unused "ctx" parameter of
bi_int_divide() has been marked with "__unused" to avoid a compilation
error.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Advertise support for TLS version 1.1, and be prepared to downgrade to
TLS version 1.0. Tested against Apache with mod_gnutls, using the
GnuTLSPriorities directive to force specific protocol versions.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Allow packet transmission to be deferred pending successful ARP
resolution. This avoids the time spent waiting for a higher-level
protocol (e.g. TCP or TFTP) to attempt retransmission.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Some PXE stacks (observed with a QLogic 8242) will always try to
prepend a link-layer header, even if the caller uses P_UNKNOWN to
indicate that the link-layer header has already been filled in. This
results in an invalid packet being transmitted.
Work around these faulty PXE stacks where possible by stripping the
existing link-layer header and allowing the PXE stack to (re)construct
the link-layer header itself.
Originally-fixed-by: Buck Huppmann <buckh@pobox.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Some iSCSI targets respond to a PDU before receiving the padding
bytes. If the target responds quickly enough, this can cause iPXE to
start processing a new TX PDU before the padding bytes have been sent,
which results in a protocol violation.
Fix by always transmitting the padding bytes along with the data
segment.
Originally-fixed-by: Shyam Iyer <shyam_iyer@dell.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
As RFC 2616 10.3.4 explains, a 303 status is the proper HTTP 1.1
behavior for what most HTTP 1.0 clients did with code 302.
Signed-off-by: Jason Lunz <lunz@acm.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Abstract out the generic line-handling portions of the syslog
putchar() routine, to allow use by other console types.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Explicitly disable the syslog console when no syslog server is
defined, rather than (ab)using the socket family address as an
equivalent console-enabled flag.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Separate out the core HTTP functionality (which is shared by both HTTP
and HTTPS) from the provision of the "http://" URI opener. This
allows for builds that support only "https://" URIs.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The PXE specification requires us to request DHCP options 128 to 135
inclusive, although these have no defined purpose.
Suggested-by: Ralf Buettner <rab@bootix.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Some iSCSI targets (observed with stgt) can be configured to reject
connections that do not use header or data digests, and will respond
with "HeaderDigest=Reject" and/or "DataDigest=Reject", while still
allowing the connection to proceed to the full feature phase.
According to a strict reading of RFC3720, we are perfectly safe to
ignore these "Reject" messages: upon such a rejection "the negotiated
key is left at its current value (or default if no value was set)".
Since the default value for both HeaderDigest and DataDigest is
"None", then the only viable conclusion to be drawn is that the value
resulting from "Reject" is still "None".
Unfortunately, stgt doesn't seem to agree with this interpretation of
events, causing us to eventually report an unhelpful "connection timed
out" message to the user when we don't get any response to our first
PDU in full feature phase.
Fix by detecting any rejected parameters and immediately reporting an
error, which at least gives the user some insight as to what the real
problem may be.
Reported-by: Michal Suchanek <hramrach@centrum.cz>
Tested-by: Michal Suchanek <hramrach@centrum.cz>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Drivers are currently expected to initialise only the hardware
address, with the link-layer protocol code taking care of converting
this into a valid link-layer address. Some drivers (e.g. undinet) can
legitimately determine both the hardware and link-layer addresses,
which may differ.
Allow for this situation by checking to see if the link-layer address
is empty before initialising it from the hardware address.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
iPXE currently uses the last four bytes of the MAC address as the DHCP
transaction identifier. Reduce the probability of collisions by
generating a random transaction identifier.
Originally-implemented-by: Amos Kong <akong@redhat.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
TCP currently neglects to allow sufficient space for its own headers
when allocating I/O buffers. This problem is masked by the fact that
the maximum link-layer header size (802.11) is substantially larger
than the common Ethernet link-layer header.
Fix by allowing sufficient space for any TCP headers, as well as the
network-layer and link-layer headers.
Reported-by: Scott K Logan <logans@cottsay.net>
Debugged-by: Scott K Logan <logans@cottsay.net>
Tested-by: Scott K Logan <logans@cottsay.net>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
timer->refcnt is allowed to be NULL, in which case the timer's
expired() method may end up freeing the timer object.
Discovered using valgrind.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
When transmitting, use the broadcast link-layer address for any
broadcast address (e.g. 192.168.0.255), not just INADDR_BROADCAST
(255.255.255.255).
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Explicitly discard any unicast packets for addresses that we do not
control, to avoid unexpected behaviour when operating in promiscuous
mode (which is now the default, thanks to FCoE).
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Allow the link layer to directly report whether or not a packet is
multicast or broadcast at the time of calling pull(), rather than
relying on heuristics to determine this at a later stage.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
According to section 14.23 of RFC2616, an HTTP Host header without
port implies the default port is used. Thus, when fetching from
anywhere but port 80 for HTTP or 443 for HTTPS, the port ought to be
explicitly given in that header. Otherwise, some servers might fail
to associate the request with the correct virtual host or generate
incorrect self-referencing URLs.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The iSCSI TX process can now be woken up by the TCP socket via
xfer_window_changed(), so it is no longer valid to assume that
iscsi_tx_step() can be called in state ISCSI_TX_IDLE only immediately
after completing a transmission.
Fix by calling iscsi_tx_done() only upon a transition into state
ISCSI_TX_IDLE.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Provide support for HTTP range requests, and expose this functionality
via the iPXE block device API. This allows SAN booting from a root
path such as:
sanboot http://boot.ipxe.org/freedos/fdfullcd.iso
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Give the step() method a pointer to the containing object, rather than
a pointer to the process. This is consistent with the operation of
interface methods, and allows a single function to serve as both an
interface method and a process step() method.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
ftp_data_deliver() does nothing except pass through the received data
to the xfer interface, and so can be eliminated by using a
pass-through interface.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
At the time of attempting ARP resolution, we already know the
transmitting network device. We can therefore record ARP errors using
netdev_tx_err() so that they show up in the output of "ifstat".
Inspired-by: Dominik Russenberger <dominik.russenberger@terreactive.ch>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Allow TX errors to be recorded against a network device even when the
packet didn't make it as far as netdev_tx().
Inspired-by: Dominik Russenberger <dominik.russenberger@terreactive.ch>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
(Ab)use the "ident" field in transmitted IPv4 packets to convey
metadata about the network device. In particular:
bits 0-3 represent the low bits of the "RX" good packet counter
bits 4-7 represent the low bits of the "RXE" bad packet counter
bits 8-15 represent the transmitted packet sequence number
This allows some relevant information about the internal state of the
network device to be read out from a packet trace from a non-debug
build of iPXE. In particular, it allows a packet trace containing
packets transmitted by iPXE to indicate whether or not any packets
have been received by iPXE.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Booting from an HTTP SAN will require HTTP range requests, which are
defined only in HTTP/1.1 and above. HTTP/1.1 mandates support for
"Transfer-Encoding: chunked", so we must support it.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Commit 3f442d3 ("[tcp] Record ts_recent on first received packet")
failed to achieve its stated intention.
Fix this (and reduce the code size) by moving the ts_recent update to
tcp_rx_seq(). This is the code responsible for advancing the window,
called by both tcp_rx_syn() and tcp_rx_data(), and so the window check
is now redundant.
Reported-by: Frank Weed <zorbustheknight@gmail.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Set the current working URI to NULL rather than to "tftp://0.0.0.0/".
Reported-by: Piotr Jaroszyński <p.jaroszynski@gmail.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
For devices that start in a link-down state, the user will see a
message such as:
[Link status: The socket is not connected (http://ipxe.org/38086001)]
Waiting for link-up on net0...
This is potentially misleading, since it suggests that there is a
genuine problem. Add a dedicated error message for "link down",
giving instead:
[Link status: Down (http://ipxe.org/38086101)]
Waiting for link-up on net0...
Reported-by: Tal Aloni <tal.aloni.il@gmail.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
netdev_close() assumes that devices that are open are on the
open_list, which wasn't true if device specific opening failed.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Commit 6861304 ("[tcp] Handle out-of-order received packets")
introduced a regression in which ts_recent would not be updated until
the first packet is received in the ESTABLISHED state, i.e. the
timestamp from the SYN+ACK packet would be ignored. This causes the
connection to be dropped by strictly-conforming TCP peers, such as
FreeBSD.
Fix by delaying the timestamp window check until after processing the
received SYN flag.
Reported-by: winders@sonnet.com
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Improve the appearance of the "config" user interface by ensuring that
settings appear in some kind of logical order.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Expose a function setting_applies() to allow a caller to determine
whether or not a particular setting is applicable to a particular
settings block.
Restrict DHCP-backed settings blocks to accepting only DHCP-based
settings.
Restrict network device settings blocks to accepting only DHCP-based
settings and network device-specific settings such as "mac".
Inspired-by: Glenn Brown <glenn@myri.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The default initiator IQN is "iqn.2000-09.org.etherboot:UNKNOWN".
This is problematic for two reasons:
a) the etherboot.org domain (and hence the associated IQN namespace)
is not under the control of the iPXE project, and
b) some targets (correctly) refuse to allow concurrent connections
from different initiators using the same initiator IQN.
Solve both problems by changing the default initiator IQN to be
iqn.2010-04.org.ipxe:<hostname> if a hostname is set, or
iqn.2010-04.org.ipxe:<uuid> if no hostname is set.
Explicit initiator IQNs set via DHCP option 203 are not affected by
this change.
Unfortunately, this change is likely to break some existing
configurations, where ACL rules have been put in place referring to
the old default initiator IQN. Users may need to update ACLs, or
force the use of the old IQN using an iPXE script line such as
set initiator-iqn iqn.2000-09.org.etherboot:UNKNOWN
or a dhcpd.conf option such as
option iscsi-initiator-iqn "iqn.2000-09.org.etherboot:UNKNOWN"
Signed-off-by: Michael Brown <mcb30@ipxe.org>
After a more accurate reading of RFC 3720, it becomes clear how NOPs
are supposed to work. The current implementation (which just ignores
NOP-Ins) is sufficient to cope with NOP-Ins sent to update CmdSN, but
will need to be extended before it can cope with NOP-Ins sent as iSCSI
keepalives.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Some iSCSI targets (observed with a Synology DS207+ NAS) send
unsolicited NOP-Ins to the initiator. RFC 3720 is remarkably unclear
and possibly self-contradictory on how NOPs are supposed to work, but
it seems as though we can legitimately just ignore any unsolicited
NOP-In PDU.
Reported-by: Marc Lecuyer <marc@maxiscreen.com>
Originally-implemented-by: Thomas Miletich <thomas.miletich@gmail.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Allow functions other than realloc() to be used to reallocate DHCP
option block data, and specify the reallocation function at the time
of calling dhcpopt_init().
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The max_len field is never used, and the len field is used only by
dhcp_tx(). Remove these two fields, and perform the necessary trivial
calculation in dhcp_tx() instead.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
For IPoIB, we currently use the hardware address (i.e. the eight-byte
GUID) as the DHCP chaddr. This works, but some PXE servers (notably
Altiris RDP) refuse to respond if the chaddr field is anything other
than six bytes in length.
We already have the notion of an Ethernet-compatible link-layer
address, which is used in the iBFT (the design of which similarly
fails to account for non-Ethernet link layers). Use this as the first
preferred alternative to the actual link-layer address when
constructing the DHCP chaddr field.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Some network cards automatically strip the VLAN header, providing the
VLAN tag via a side channel such as a completion queue entry. These
cards need to be able to report receive completions directly against
the relevant VLAN device.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
VLAN device names have the form "netX.Y", e.g. "net0.5" for VLAN 5 on
net0. This use of "." conflicts with the use of "." as the
hierarchical separator in settings block names, with the result that
VLAN device settings cannot be accessed by name.
It would be trivial to treat the VLAN device settings as being a child
of the trunk device settings, but this would cause the VLAN device
settings to be applied to the trunk device: for example, setting
"net0.5/ip" would then apply the IP address to both net0.5 and net0.
Fix by changing the VLAN device name to use "-" instead of ".": the
VLAN device "net0.5" is now "net0-5".
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Pass the settings block name as a parameter to register_settings(),
rather than defining it with settings_init() (and then possibly
changing it by directly manipulating settings->name).
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Michael Brown