Question
I am building a URL fetcher in Go that sends an HTTP GET request for each URL:
resp, err := http.Get(url)
How can I set a custom timeout for every GET request? The default behavior can wait too long, which makes the fetcher slow. I want a request to stop after approximately 40–45 seconds, report that it timed out, and then continue to the next URL.
Short Answer
By the end of this page, you will know why http.Get has no timeout setting of its own, how to configure a reusable http.Client with a total request timeout, how to detect timeout errors, and why response bodies must always be closed.
Concept
http.Get(url) is a convenient shortcut that uses Go's shared http.DefaultClient. That default client does not set a total request deadline, so a slow server, stalled connection, or response body that never finishes can make your program wait for a very long time.
To set a timeout, create an http.Client and set its Timeout field:
client := &http.Client{
Timeout: 45 * time.Second,
}
Client.Timeout is a deadline for the entire HTTP exchange. It includes:
- Connecting to the server
- Following redirects
- Waiting for response headers
- Reading the response body
If that deadline expires, the request is canceled and the client returns an error.
In real programs, create a configured client once and reuse it. Reusing the client lets Go reuse network connections efficiently and gives every request the same safety rules.
Mental Model
Think of an HTTP request as a delivery trip with one overall deadline.
Without a timeout, the driver can wait forever at a blocked road, a closed gate, or a warehouse that never finishes loading. http.Client.Timeout gives the entire trip a 45-second deadline. When time runs out, Go cancels the trip so your fetcher can continue with the next URL.
A reusable http.Client is like using one well-maintained delivery vehicle for all trips instead of buying a new vehicle for every address.
Syntax and Examples
Create a client with Timeout, then call its Get method instead of the package-level http.Get function.
package main
import (
"fmt"
"net/http"
"time"
)
func main() {
client := &http.Client{
Timeout: 45 * time.Second,
}
resp, err := client.Get("https://example.com")
if err != nil {
fmt.Println("request failed:", err)
return
}
defer resp.Body.Close()
fmt.Println("status:", resp.Status)
}
45 * time.Second creates a time.Duration. The client stops the request if the complete operation exceeds that duration.
For a URL fetcher, create the client before the loop:
client := &http.Client{Timeout: 45 * time.Second}
for _, url := range urls {
resp, err := client.Get(url)
if err != nil {
fmt.Printf("%s: %v\n", url, err)
continue
}
resp.Body.Close()
}
Closing resp.Body is essential, even when you do not need the response content.
Step by Step Execution
Consider this program:
client := &http.Client{Timeout: 40 * time.Second}
resp, err := client.Get("https://example.com")
if err != nil {
fmt.Println("could not fetch URL:", err)
return
}
defer resp.Body.Close()
fmt.Println(resp.StatusCode)
Execution flow:
http.Clientis created with a 40-second total deadline.client.Getbuilds and sends an HTTP GET request.- Go connects to the host, sends the request, and waits for the server's response.
- If all work finishes before 40 seconds,
respcontains the response anderrisnil. - If connecting, redirecting, waiting, or reading takes too long, Go cancels the request and
erris non-nil. - When a response exists,
defer resp.Body.Close()schedules the body to be closed when the surrounding function returns. This releases resources and allows connections to be reused.
When you read the body, the same timeout still applies:
body, err := io.ReadAll(resp.Body)
if err != nil {
// This can fail if the client deadline expires while reading.
return
}
_ = body
Real World Use Cases
HTTP timeouts prevent one unavailable dependency from stopping an entire program.
- Web crawlers and URL checkers: Skip dead or extremely slow pages and continue checking the remaining URLs.
- API clients: Fail promptly when a payment, weather, or authentication API is unresponsive.
- Background workers: Avoid workers becoming stuck forever on one job.
- Microservices: Put a limit on calls to downstream services so your service can return an error within its own deadline.
- Data import scripts: Record failed URLs and continue importing data from other sources.
The right timeout depends on the operation. A small health-check request may use a few seconds, while downloading a large file may need a longer deadline.
Real Codebase Usage
In production code, configure and reuse a client rather than calling http.Get directly.
Reuse one configured client
var fetchClient = &http.Client{
Timeout: 45 * time.Second,
}
A shared client is safe for concurrent use and can reuse connections.
Use a per-request context when a caller controls the deadline
A context is useful when different requests need different limits, or when a parent operation can be canceled.
ctx, cancel := context.WithTimeout(context.Background(), 45*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return err
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
Check HTTP status separately from transport errors
A timeout is returned as err. An HTTP status such as 404 or 500 normally still returns a non-nil resp and a nil transport error.
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return fmt.Errorf("unexpected status: %s", resp.Status)
}
Handle timeout errors intentionally
if err != nil {
if errors.Is(err, context.DeadlineExceeded) {
fmt.Println("request timed out")
} else {
fmt.Println("request failed:", err)
}
}
Use the context and errors packages for this check. Error wrapping can vary, so avoid relying only on matching an error's text.
Common Mistakes
Continuing to use http.Get
This uses the default client, not your configured one:
client := &http.Client{Timeout: 45 * time.Second}
resp, err := http.Get(url) // Wrong: client is ignored
Use client.Get(url) instead.
resp, err := client.Get(url)
Creating a client but forgetting to set Timeout
client := &http.Client{}
This still has no total timeout. Set Timeout explicitly, or use a request context with a deadline.
Forgetting to close the response body
resp, err := client.Get(url)
if err != nil {
return
}
fmt.Println(resp.Status)
// Broken: resp.Body is never closed.
Fix it immediately after checking err:
defer resp.Body.Close()
In a long loop, putting defer directly inside the loop delays closing every body until the outer function returns. Prefer a helper function that defers the close per request, or close the body explicitly after reading it.
Treating an HTTP error status as a Go error
A 404 Not Found response usually does not make client.Get return an error. Check resp.StatusCode yourself.
Using error strings to identify timeouts
Avoid code such as:
if err.Error() == "request timed out" {
// fragile
}
Use errors.Is(err, context.DeadlineExceeded) when appropriate, or inspect a wrapped net.Error when you need lower-level network error handling.
Assuming Client.Timeout only covers connecting
It covers the complete request, including reading the body. If you download large files, choose a timeout that fits the expected transfer time or use more specialized transport and context settings.
Comparisons
| Approach | What it limits | Best use |
|---|---|---|
http.Get(url) | No configured total timeout on the default client | Quick experiments only; usually avoid in robust applications |
http.Client{Timeout: ...} | The entire request: connection, redirects, headers, and body reading | A consistent timeout for a reusable client |
context.WithTimeout with NewRequestWithContext | One specific request, and it can inherit cancellation from a parent context | Request-specific deadlines and cancellation |
http.Transport timeouts | Individual phases such as TLS handshake or waiting for headers | Fine-grained client behavior when a total timeout alone is insufficient |
http.Client.Timeout is usually the simplest choice for a URL fetcher where every URL should have the same 40–45 second limit. A context is often better when a function receives a context.Context from its caller.
Cheat Sheet
import (
"context"
"errors"
"net/http"
"time"
)
client := &http.Client{
Timeout: 45 * time.Second,
}
resp, err := client.Get(url)
if err != nil {
if errors.Is(err, context.DeadlineExceeded) {
// Timed out
}
// Handle other errors
}
if resp != nil {
defer resp.Body.Close()
}
http.Getuseshttp.DefaultClient; it does not use your custom client.- Set
client.Timeoutfor a total request deadline. - Reuse
http.Clientinstances instead of creating one per request. - Always close a successful response body.
- A timeout is a Go error; a
4xxor5xxHTTP response is usually not. - For a request-specific deadline, use
context.WithTimeoutandhttp.NewRequestWithContext. - Choose units from the
timepackage:time.Second,time.Millisecond, andtime.Minute.
FAQ
How do I set a timeout for http.Get in Go?
Create an http.Client with a Timeout and call client.Get(url). The package-level http.Get function does not accept a timeout argument.
Does http.Client.Timeout include reading the response body?
Yes. It limits the full operation, including connecting, redirects, waiting for headers, and reading the response body.
Is a 45-second timeout written as 45 * time.Second?
Yes. Import time and write Timeout: 45 * time.Second.
How can I tell whether a Go HTTP request timed out?
Check the returned error, commonly with errors.Is(err, context.DeadlineExceeded). Do not rely on exact error-message text.
Should I create a new http.Client for every URL?
Usually no. Create one configured client and reuse it. Reuse improves connection reuse and keeps configuration consistent.
Why must I close resp.Body?
Closing the body releases resources and helps Go reuse the underlying network connection. Not closing bodies can eventually exhaust resources.
Does a 500 status code cause client.Get to return an error?
Usually not. A server response with status 500 gives you a response object. Check resp.StatusCode to decide whether the result is acceptable.
Should I use a client timeout or a context timeout?
Use Client.Timeout for a simple, consistent limit across all requests from one client. Use a context when an individual request needs its own deadline or must be canceled by its caller.
Mini Project
Description
Build a small sequential URL checker. It fetches each URL with one reusable HTTP client, prints the status for successful requests, identifies timeouts, and continues after failures. This is the basic behavior needed by crawlers, monitoring scripts, and import tools.
Goal
Fetch each URL with a 45-second limit without allowing one slow URL to stop the entire list.
Requirements
Requirement 1 Requirement 2 Requirement 3
Keep learning
Related questions
Are Go Maps Passed by Value or Reference?
Learn Go map parameter and return semantics: maps are passed by value, while copies share underlying data. Know when map pointers are useful.
Authentication for Go REST APIs: Sessions, Tokens, and Middleware
Learn practical authentication patterns for Go REST APIs, including password hashing, server sessions, bearer tokens, middleware, and security rules.
Automatic Build Versioning in Go: Embed Incrementing Build Numbers
Learn how to add automatic build versioning in Go using linker flags, build metadata, CI counters, and Git-based version values.