Amish Kushwaha - October 26, 2026
btp-go Part 4 — Worked Example: Counting Files in AL11 over SFTP
Part 4 of 5 in the btp-go series. Part 1 · Part 2 · Part 3
Every SAP ABAP system has a filesystem you can browse through transaction AL11. Application servers accumulate files there: spool outputs, IDocs, interface files from legacy batch jobs, log files from custom programs. Periodically a monitoring application needs to check whether files exist, count them, move them, or read their contents.
The traditional approach is RFC: a custom ABAP function module that reads the filesystem and returns results. RFC works, but it requires ABAP development, a transport, and an RFC destination in your BTP integration. An alternative that requires none of that is SFTP: if the ABAP application server has an SSH daemon running (most do in modern SAP Basis configurations), you can reach it through Cloud Connector and use the standard SFTP protocol to navigate AL11 paths.
This post builds a complete Cloud Foundry application that does exactly that. It uses sshclient.Dial for the high-level approach, explains when you might want the lower-level alternative, and covers the deployment.
What You Need on the SAP Side
Before writing any Go code, confirm the SAP configuration:
SSH daemon on the ABAP server. The OS-level sshd must be running and listening on a port. The default is 22, but some Basis teams move it to a non-standard port. Check with your Basis team.
Cloud Connector virtual host mapping. In the Cloud Connector administration UI (typically at https://your-scc-host:8443), add an on-premises resource mapping:
- Internal host: the actual hostname or IP of the ABAP application server
- Internal port: the SSH port (usually 22)
- Virtual host: the name your BTP app will use, e.g.,
my-abap-dev - Virtual port: 22
An SSH destination in BTP. Create a destination in your BTP subaccount with:
- Type: TCP (not HTTP — you are opening a raw TCP tunnel, not an HTTP connection)
- Proxy Type: OnPremise
- Host: the virtual host you configured in Cloud Connector (
my-abap-dev) - Port: 22
- Authentication: BasicAuthentication (for username/password) or NoAuthentication (if using an SSH key)
- User/Password or the
sshKeyadditional property
Service bindings. Your CF app needs bindings for xsuaa, connectivity, and destination services.
Two Approaches: Low-Level and High-Level
The btp-go repository ships two examples for SFTP. They represent different points on the control-vs-convenience spectrum.
examples/sftp-count is the low-level approach. It creates a connectivity.Dialer, calls dialer.Dial to get a net.Conn, then manually builds an ssh.ClientConfig and calls ssh.NewClientConn. There is no retry logic. It shows you every step of the stack and is useful when you need precise control — for example, when you want to implement your own connection pooling, or when you need to use an SSH certificate rather than a simple password.
examples/sftp-sshclient is the high-level approach using sshclient.Dial. It handles the ssh.ClientConfig construction from a Destination, implements exponential-backoff retry for transient connection failures, and returns a ready-to-use *ssh.Client. This is the right choice for most new code.
For a production monitoring application, use the high-level approach. The rest of this post walks through it in detail.
The Data Flow
┌───────────────────────────────────────────────────────────────────────────┐
│ Cloud Foundry │
│ │
│ ┌──────────────┐ Token ┌──────────────────┐ │
│ │ CF App │───────────▶│ XSUAA │ │
│ │ (btp-go) │◀───────────│ (token endpoint)│ │
│ │ │ JWT └──────────────────┘ │
│ │ │ │
│ │ │ Find() ┌──────────────────┐ │
│ │ │───────────▶│ Destination Svc │ │
│ │ │◀───────────│ │ │
│ │ │ Dest cfg └──────────────────┘ │
│ │ │ │
│ │ │ SOCKS5 ┌──────────────────────────────────────────┐ │
│ │ │───────────▶│ Connectivity Proxy (port 20004) │ │
│ └──────────────┘ 0x80 └────────────────────┬─────────────────────┘ │
└───────────────────────────────────────────────────┼───────────────────────┘
│ TLS tunnel
┌───────────────────────────────────────────────────┼───────────────────────┐
│ Corporate Network │ │
│ ┌──────▼─────────────┐ │
│ │ Cloud Connector │ │
│ │ (on-prem agent) │ │
│ └──────┬─────────────┘ │
│ │ TCP port 22 │
│ ┌──────▼─────────────┐ │
│ │ ABAP App Server │ │
│ │ sshd + AL11 files │ │
│ └────────────────────┘ │
└───────────────────────────────────────────────────────────────────────────┘The Complete Application
The application exposes a single HTTP endpoint. When called, it connects to the ABAP server over SFTP and returns a count of files in a directory.
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"log"
"net/http"
"os"
"time"
"github.com/bluefunda/btp-go/binding"
"github.com/bluefunda/btp-go/binding/auto"
"github.com/bluefunda/btp-go/connectivity"
"github.com/bluefunda/btp-go/destination"
"github.com/bluefunda/btp-go/sshclient"
"github.com/bluefunda/btp-go/xsuaa"
"github.com/pkg/sftp"
)
// app holds the long-lived BTP clients. These are initialized once at startup
// and shared across all HTTP requests. They are safe for concurrent use.
type app struct {
dialer *connectivity.Dialer
destClient *destination.Client
}
func main() {
ctx := context.Background()
a, err := newApp(ctx)
if err != nil {
log.Fatalf("startup failed: %v", err)
}
http.HandleFunc("/count", a.countHandler)
port := os.Getenv("PORT")
if port == "" {
port = "8080"
}
log.Printf("listening on :%s", port)
if err := http.ListenAndServe(":"+port, nil); err != nil {
log.Fatal(err)
}
}
func newApp(ctx context.Context) (*app, error) {
// Auto-detect CF or Kyma and load service bindings.
prov, err := auto.NewProvider()
if err != nil {
return nil, fmt.Errorf("service binding detection: %w", err)
}
// XSUAA: build a cached, concurrent-safe token source.
xsuaaB, err := binding.XSUAA(prov, "")
if err != nil {
return nil, fmt.Errorf("XSUAA binding: %w", err)
}
ts := xsuaa.NewClientCredentialsSource(xsuaa.Config{
ClientID: xsuaaB.ClientID,
ClientSecret: xsuaaB.ClientSecret,
TokenURL: xsuaaB.TokenURL,
Skew: 90,
})
// Connectivity: build the SOCKS5 dialer.
connB, err := binding.Connectivity(prov, "")
if err != nil {
return nil, fmt.Errorf("Connectivity binding: %w", err)
}
dialer, err := connectivity.NewDialer(connectivity.Config{
ProxyHost: connB.ProxyHost,
ProxyPort: connB.ProxyPort,
TokenSource: ts,
DialTimeout: 30 * time.Second,
})
if err != nil {
return nil, fmt.Errorf("connectivity dialer: %w", err)
}
// Destination: build the lookup client.
destB, err := binding.Destination(prov, "")
if err != nil {
return nil, fmt.Errorf("Destination binding: %w", err)
}
destClient := destination.NewClient(destB.ServiceURL, ts, http.DefaultClient)
return &app{dialer: dialer, destClient: destClient}, nil
}
type countResponse struct {
Directory string `json:"directory"`
Count int `json:"count"`
DurationMs int64 `json:"duration_ms"`
}
func (a *app) countHandler(w http.ResponseWriter, r *http.Request) {
dir := r.URL.Query().Get("dir")
if dir == "" {
dir = "/usr/sap/tmp" // a common AL11 path
}
destName := r.URL.Query().Get("dest")
if destName == "" {
destName = "my-abap-sftp"
}
start := time.Now()
count, err := a.countFiles(r.Context(), destName, dir)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(countResponse{
Directory: dir,
Count: count,
DurationMs: time.Since(start).Milliseconds(),
})
}
func (a *app) countFiles(ctx context.Context, destName, dir string) (int, error) {
// Step 1: Resolve the destination — one HTTPS call to the Destination Service.
dest, err := a.destClient.Find(ctx, destName)
if err != nil {
return 0, fmt.Errorf("destination lookup %q: %w", destName, err)
}
// Step 2: Open an SSH client through Cloud Connector.
// sshclient.Dial handles:
// - Building ssh.ClientConfig from the Destination fields
// - Calling a.dialer.Dial to get the net.Conn
// - Retrying on transient handshake failures
sshCfg := sshclient.Config{
Dialer: a.dialer,
RetryOpts: sshclient.RetryOpts{
MaxAttempts: 5,
BaseDelay: 500 * time.Millisecond,
Jitter: true,
},
}
sshClient, err := sshclient.Dial(ctx, sshCfg, dest)
if err != nil {
return 0, fmt.Errorf("SSH dial to %s: %w", dest.Host, err)
}
defer sshClient.Close()
// Step 3: Open an SFTP client over the SSH connection.
sftpClient, err := sftp.NewClient(sshClient)
if err != nil {
return 0, fmt.Errorf("SFTP client: %w", err)
}
defer sftpClient.Close()
// Step 4: List the directory.
entries, err := sftpClient.ReadDir(dir)
if err != nil {
return 0, fmt.Errorf("ReadDir %q: %w", dir, err)
}
return len(entries), nil
}The code is about 130 lines including imports and comments. There are no goroutine pools, no connection caches, no background refresh goroutines in application code — the library handles all of that.
The Retry Problem: sshd MaxStartups
There is an operational detail that bites every team that runs high-concurrency SSH workloads: the MaxStartups parameter in sshd’s configuration.
MaxStartups controls how many unauthenticated SSH connections sshd accepts simultaneously. Its default value on most Linux distributions is 10:30:100, which means: start dropping connections randomly when there are 10 unauthenticated sessions in progress, with a 30% probability, rising linearly to 100% when there are 100 pending sessions.
In a Cloud Foundry environment with multiple app instances or a monitoring tool that starts up multiple goroutines simultaneously, you can easily hit MaxStartups. When sshd drops your connection, the client sees a sudden EOF or connection reset during the SSH handshake — not a clean error.
sshclient.RetryOpts handles this:
RetryOpts: sshclient.RetryOpts{
MaxAttempts: 5,
BaseDelay: 500 * time.Millisecond,
Jitter: true,
},MaxAttempts=5means up to 5 total attempts (1 initial + 4 retries).BaseDelay=500msis the starting delay. Each retry doubles it: 500ms, 1s, 2s, 4s.Jitter=trueadds a random fraction of the delay to each wait, so that multiple goroutines that failed simultaneously don’t all retry at the exact same instant (which would just cause the same problem again).
sshclient distinguishes transient failures (handshake EOF during MaxStartups drops) from permanent failures (authentication rejected, host key mismatch). It only retries transient ones. A wrong password fails immediately after one attempt.
If you are using the low-level approach (examples/sftp-count), you have to implement this yourself. The high-level sshclient.Dial gives it to you for free.
Deploying to Cloud Foundry
The manifest.yml for the application:
---
applications:
- name: al11-counter
memory: 64M
disk_quota: 128M
instances: 2
buildpacks:
- go_buildpack
env:
GOPACKAGENAME: github.com/yourorg/al11-counter
GOVERSION: go1.25.10
services:
- my-xsuaa
- my-connectivity
- my-destinationThe service names (my-xsuaa, my-connectivity, my-destination) must match the service instances you created in your BTP subaccount. When CF pushes the application, it injects VCAP_SERVICES containing credentials for all three services. binding/auto picks them up automatically.
cf pushThat’s the entire deployment. The Go buildpack compiles the application, CF injects the service bindings, and the app starts.
With instances: 2, CF runs two copies of your app. Both share the same service bindings. Both create their own connectivity.Dialer and destination.Client at startup. Each instance independently caches XSUAA tokens — there is no shared token cache across instances, which is fine because the token endpoint can serve both.
Testing the Endpoint
# Count files in the default directory
curl https://al11-counter.cfapps.eu10.hana.ondemand.com/count
# Count files in a specific AL11 path
curl "https://al11-counter.cfapps.eu10.hana.ondemand.com/count?dir=/usr/sap/SID/DVEBMGS00/log&dest=my-abap-sftp"Response:
{
"directory": "/usr/sap/SID/DVEBMGS00/log",
"count": 47,
"duration_ms": 312
}312ms end-to-end: that includes the Destination Service HTTP lookup, the SOCKS5 tunnel negotiation, the SSH handshake, and the SFTP ReadDir call. For a monitoring endpoint that runs every few minutes, that is perfectly acceptable.
When to Use the Low-Level Approach
sshclient.Dial is the right choice for most use cases. The low-level sftp-count example exists for cases where you need things that sshclient does not provide:
Certificate-based authentication. sshclient reads User, Password, and Properties["sshKey"] from the Destination. If you need to use an SSH certificate (signed key) rather than a raw private key, you need to build the ssh.ClientConfig yourself.
Connection pooling. sshclient.Dial opens a new connection for each call. If you are making high-frequency SFTP calls — more than a few per second — you will want to maintain a pool of SSH connections. The low-level approach gives you the *ssh.Client directly, which you can store and reuse.
Custom host key verification. By default sshclient uses ssh.InsecureIgnoreHostKey() when no host key is configured in the destination. If you want strict host key checking (you should in production), you need to build the ssh.ClientConfig yourself and supply a HostKeyCallback that checks against a known-hosts file or a pinned key from the destination’s Properties map.
Non-SFTP SSH operations. *ssh.Client supports running commands (client.NewSession().Run(cmd)) and forwarding ports, not just SFTP. The low-level approach is the right interface when the terminal goal is anything other than a file transfer.
Key Lessons
A few things stood out when building this:
Cloud Connector virtual hosts are not DNS names. They exist only inside the Cloud Connector mapping table. dialer.Dial("my-abap-dev", "22", "") will not work unless my-abap-dev is configured in Cloud Connector — there is no DNS fallback, no resolution step. This is a common source of confusion when first setting things up.
The Destination Service is not a cache. It fetches the destination configuration from a central store on every Find() call. In v1.0.0 of btp-go the destination module does not cache results. For a monitoring app that checks files every minute, this is fine — one HTTP call per check. For an app that makes hundreds of destination lookups per second, you would want to cache the *Destination result in your application layer with a reasonable TTL (5-10 minutes is typical).
SFTP sessions are cheap relative to SSH connections. Opening the SSH connection (TCP + SOCKS5 + SSH handshake) is the expensive part — a few hundred milliseconds. Once you have an *ssh.Client, opening an sftp.Client on top of it takes single-digit milliseconds. If you need to access multiple directories in one request, keep the sshClient open and open multiple SFTP clients on it.
The sftp package is not in the Go standard library. It is github.com/pkg/sftp, the de facto standard Go SFTP client. btp-go does not import it (that would force the dependency on all users), but your application will. Pin the version in your go.mod.
What’s Next
Part 4 covered TCP destinations: SSH, SFTP, the SOCKS5 tunnel. Part 5 covers HTTP destinations: SAP Gateway, OData v2, CSRF tokens, and the httpclient module that wires authentication token injection, on-prem routing, and session-affinity cookies into a standard *http.Client.
Next up — Part 5: Worked Example: HTTP Destinations and OData Writes.
Ready to transform your SAP experience?
Join thousands of developers using BlueFunda AI tools.