RustFS使用指南

Thu, 16 Apr 2026 00:00:00 +0000

什么是 RustFS？

RustFS 是一个基于 Rust 实现的高性能 S3 兼容对象存储服务。在本项目中，我们使用 AWS SDK v2 访问 RustFS，享受与 AWS S3 完全兼容的 API。RustFS 提供了：

S3 兼容 API：使用标准的 S3 协议，可与现有 S3 工具无缝集成
高性能：Rust 语言实现，保证低延迟和高吞吐量
简单部署：轻量级部署，适合私有化部署场景
成本效益：适合中小规模数据存储

项目集成

Maven 依赖

<properties>
    <aws-sdk.version>2.29.0</aws-sdk.version>
</properties>

<dependencies>
    <!-- AWS S3 SDK -->
    <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>s3</artifactId>
        <version>${aws-sdk.version}</version>
    </dependency>
</dependencies>

S3 客户端配置

import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.net.URI;

@Configuration
@RequiredArgsConstructor
public class S3Config {

    private final StorageConfigProperties storageConfig;

    @Bean
    public S3Client s3Client() {
        AwsBasicCredentials credentials = AwsBasicCredentials.create(
            storageConfig.getAccessKey(),
            storageConfig.getSecretKey()
        );

        return S3Client.builder()
            .endpointOverride(URI.create(storageConfig.getEndpoint()))
            .region(Region.of(storageConfig.getRegion()))
            .credentialsProvider(StaticCredentialsProvider.create(credentials))
            // 关键配置：使用路径风格访问，避免 DNS 解析问题
            .forcePathStyle(true)
            .build();
    }
}

配置文件

storage:
  endpoint: http://localhost:9000
  region: us-east-1
  bucket: my-bucket
  access-key: your-access-key
  secret-key: your-secret-key

基础使用

文件上传

import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.*;

@Service
@RequiredArgsConstructor
public class FileStorageService {

    private final S3Client s3Client;
    private final StorageConfigProperties storageConfig;

    public String uploadFile(MultipartFile file, String prefix) {
        String fileKey = generateFileKey(file.getOriginalFilename(), prefix);

        PutObjectRequest putRequest = PutObjectRequest.builder()
            .bucket(storageConfig.getBucket())
            .key(fileKey)
            .contentType(file.getContentType())
            .contentLength(file.getSize())
            .build();

        try {
            s3Client.putObject(putRequest,
                RequestBody.fromInputStream(file.getInputStream(), file.getSize()));
            return fileKey;
        } catch (IOException e) {
            throw new BusinessException("文件上传失败");
        }
    }

    public String uploadResume(MultipartFile file) {
        return uploadFile(file, "resumes");
    }
}

文件下载

public byte[] downloadFile(String fileKey) {
    if (!fileExists(fileKey)) {
        throw new BusinessException("文件不存在: " + fileKey);
    }

    GetObjectRequest getRequest = GetObjectRequest.builder()
        .bucket(storageConfig.getBucket())
        .key(fileKey)
        .build();

    try {
        return s3Client.getObjectAsBytes(getRequest).asByteArray();
    } catch (S3Exception e) {
        throw new BusinessException("文件下载失败: " + e.getMessage());
    }
}

文件删除

public void deleteFile(String fileKey) {
    if (fileKey == null || fileKey.isEmpty()) {
        return;
    }

    if (!fileExists(fileKey)) {
        log.warn("文件不存在，跳过删除: {}", fileKey);
        return;
    }

    DeleteObjectRequest deleteRequest = DeleteObjectRequest.builder()
        .bucket(storageConfig.getBucket())
        .key(fileKey)
        .build();

    s3Client.deleteObject(deleteRequest);
}

文件是否存在检查

public boolean fileExists(String fileKey) {
    try {
        HeadObjectRequest headRequest = HeadObjectRequest.builder()
            .bucket(storageConfig.getBucket())
            .key(fileKey)
            .build();
        s3Client.headObject(headRequest);
        return true;
    } catch (NoSuchKeyException e) {
        return false;
    } catch (S3Exception e) {
        log.warn("检查文件存在性失败: {} - {}", fileKey, e.getMessage());
        return false;
    }
}

获取文件信息

public long getFileSize(String fileKey) {
    HeadObjectRequest headRequest = HeadObjectRequest.builder()
        .bucket(storageConfig.getBucket())
        .key(fileKey)
        .build();

    return s3Client.headObject(headRequest).contentLength();
}

public String getFileUrl(String fileKey) {
    return String.format("%s/%s/%s",
        storageConfig.getEndpoint(),
        storageConfig.getBucket(),
        fileKey);
}

文件键设计

按日期和 UUID 组织文件

private String generateFileKey(String originalFilename, String prefix) {
    LocalDateTime now = LocalDateTime.now();
    String datePath = now.format(DateTimeFormatter.ofPattern("yyyy/MM/dd"));
    String uuid = UUID.randomUUID().toString().substring(0, 8);
    String safeName = sanitizeFilename(originalFilename);
    return String.format("%s/%s/%s_%s", prefix, datePath, uuid, safeName);
}

生成的路径示例：resumes/2026/04/16/a1b2c3d4_张三简历.pdf

中文文件名处理

将中文文件名转换为拼音，保护特殊字符：

import net.sourceforge.pinyin4j.PinyinHelper;
import net.sourceforge.pinyin4j.format.HanyuPinyinCaseType;
import net.sourceforge.pinyin4j.format.HanyuPinyinOutputFormat;
import net.sourceforge.pinyin4j.format.exception.BadHanyuPinyinOutputFormatCombination;

private String sanitizeFilename(String filename) {
    if (filename == null || filename.isEmpty()) {
        return "unknown";
    }
    return convertToPinyin(filename);
}

private String convertToPinyin(String input) {
    HanyuPinyinOutputFormat format = new HanyuPinyinOutputFormat();
    format.setCaseType(HanyuPinyinCaseType.LOWERCASE);
    format.setToneType(HanyuPinyinToneType.WITHOUT_TONE);

    StringBuilder result = new StringBuilder();
    for (char ch : input.toCharArray()) {
        try {
            String[] pinyins = PinyinHelper.toHanyuPinyinStringArray(ch, format);
            if (pinyins != null && pinyins.length > 0) {
                // 首字母大写（大驼峰）
                result.append(capitalize(pinyins[0]));
            } else {
                result.append(sanitizeChar(ch));
            }
        } catch (BadHanyuPinyinOutputFormatCombination e) {
            result.append(sanitizeChar(ch));
        }
    }
    return result.toString();
}

private char sanitizeChar(char ch) {
    // 只保留字母、数字、点号、下划线、连字符
    if ((ch >= 'a' && ch <= 'z') || (ch >= 'A' && ch <= 'Z')
            || (ch >= '0' && ch <= '9') || ch == '.' || ch == '_' || ch == '-') {
        return ch;
    }
    return '_';
}

存储桶操作

确保存储桶存在

public void ensureBucketExists() {
    try {
        HeadBucketRequest headRequest = HeadBucketRequest.builder()
            .bucket(storageConfig.getBucket())
            .build();
        s3Client.headBucket(headRequest);
        log.info("存储桶已存在: {}", storageConfig.getBucket());
    } catch (NoSuchBucketException e) {
        log.info("存储桶不存在，正在创建: {}", storageConfig.getBucket());
        CreateBucketRequest createRequest = CreateBucketRequest.builder()
            .bucket(storageConfig.getBucket())
            .build();
        s3Client.createBucket(createRequest);
        log.info("存储桶创建成功: {}", storageConfig.getBucket());
    } catch (S3Exception e) {
        log.error("检查存储桶失败: {}", e.getMessage());
    }
}

列出存储桶中的对象

public List<S3Object> listObjects(String prefix) {
    ListObjectsV2Request listRequest = ListObjectsV2Request.builder()
        .bucket(storageConfig.getBucket())
        .prefix(prefix)
        .build();

    ListObjectsV2Response response = s3Client.listObjectsV2(listRequest);
    return response.contents();
}

完整服务封装

以下是项目中实际使用的文件存储服务：

@Slf4j
@Service
@RequiredArgsConstructor
public class FileStorageService {

    private final S3Client s3Client;
    private final StorageConfigProperties storageConfig;

    public String uploadResume(MultipartFile file) {
        return uploadFile(file, "resumes");
    }

    public String uploadKnowledgeBase(MultipartFile file) {
        return uploadFile(file, "knowledgebases");
    }

    public void deleteResume(String fileKey) {
        deleteFile(fileKey);
    }

    public void deleteKnowledgeBase(String fileKey) {
        deleteFile(fileKey);
    }

    public byte[] downloadFile(String fileKey) {
        if (!fileExists(fileKey)) {
            throw new BusinessException("文件不存在: " + fileKey);
        }

        GetObjectRequest getRequest = GetObjectRequest.builder()
            .bucket(storageConfig.getBucket())
            .key(fileKey)
            .build();
        return s3Client.getObjectAsBytes(getRequest).asByteArray();
    }

    private String uploadFile(MultipartFile file, String prefix) {
        String originalFilename = file.getOriginalFilename();
        String fileKey = generateFileKey(originalFilename, prefix);

        PutObjectRequest putRequest = PutObjectRequest.builder()
            .bucket(storageConfig.getBucket())
            .key(fileKey)
            .contentType(file.getContentType())
            .contentLength(file.getSize())
            .build();

        try {
            s3Client.putObject(putRequest,
                RequestBody.fromInputStream(file.getInputStream(), file.getSize()));
            log.info("文件上传成功: {} -> {}", originalFilename, fileKey);
            return fileKey;
        } catch (IOException e) {
            throw new BusinessException("文件读取失败");
        } catch (S3Exception e) {
            throw new BusinessException("文件存储失败: " + e.getMessage());
        }
    }

    private void deleteFile(String fileKey) {
        if (fileKey == null || fileKey.isEmpty()) {
            return;
        }

        if (!fileExists(fileKey)) {
            log.warn("文件不存在，跳过删除: {}", fileKey);
            return;
        }

        DeleteObjectRequest deleteRequest = DeleteObjectRequest.builder()
            .bucket(storageConfig.getBucket())
            .key(fileKey)
            .build();
        s3Client.deleteObject(deleteRequest);
        log.info("文件删除成功: {}", fileKey);
    }

    public boolean fileExists(String fileKey) {
        try {
            HeadObjectRequest headRequest = HeadObjectRequest.builder()
                .bucket(storageConfig.getBucket())
                .key(fileKey)
                .build();
            s3Client.headObject(headRequest);
            return true;
        } catch (NoSuchKeyException e) {
            return false;
        } catch (S3Exception e) {
            log.warn("检查文件存在性失败: {} - {}", fileKey, e.getMessage());
            return false;
        }
    }

    public String getFileUrl(String fileKey) {
        return String.format("%s/%s/%s",
            storageConfig.getEndpoint(),
            storageConfig.getBucket(),
            fileKey);
    }

    public void ensureBucketExists() {
        // 详见前文的实现
    }

    private String generateFileKey(String originalFilename, String prefix) {
        LocalDateTime now = LocalDateTime.now();
        String datePath = now.format(DateTimeFormatter.ofPattern("yyyy/MM/dd"));
        String uuid = UUID.randomUUID().toString().substring(0, 8);
        String safeName = sanitizeFilename(originalFilename);
        return String.format("%s/%s/%s_%s", prefix, datePath, uuid, safeName);
    }

    private String sanitizeFilename(String filename) {
        // 使用拼音转换实现
    }
}

错误处理

常见错误处理

try {
    s3Client.putObject(putRequest, RequestBody.fromInputStream(inputStream, size));
} catch (NoSuchBucketException e) {
    // 存储桶不存在
    throw new BusinessException("存储桶不存在: " + storageConfig.getBucket());
} catch (NoSuchKeyException e) {
    // 文件不存在
    throw new BusinessException("文件不存在: " + fileKey);
} catch (S3Exception e) {
    // 其他 S3 错误
    log.error("S3 操作失败: {} - {}", e.awsErrorDetails().errorCode(),
              e.awsErrorDetails().errorMessage());
    throw new BusinessException("S3 操作失败: " + e.getMessage());
} catch (IOException e) {
    // IO 错误
    throw new BusinessException("文件读取失败");
}

AWS SDK v2 常用操作一览

上传下载

// 上传文件
s3Client.putObject(PutObjectRequest.builder()
    .bucket(bucket)
    .key(key)
    .contentType(contentType)
    .contentLength(size)
    .build(),
    RequestBody.fromInputStream(inputStream, size));

// 下载文件
byte[] data = s3Client.getObjectAsBytes(GetObjectRequest.builder()
    .bucket(bucket)
    .key(key)
    .build()).asByteArray();

// 删除文件
s3Client.deleteObject(DeleteObjectRequest.builder()
    .bucket(bucket)
    .key(key)
    .build());

存储桶操作

// 创建存储桶
s3Client.createBucket(CreateBucketRequest.builder()
    .bucket(bucketName)
    .build());

// 列出对象
s3Client.listObjectsV2(ListObjectsV2Request.builder()
    .bucket(bucket)
    .prefix(prefix)
    .build());

// 获取对象元数据
s3Client.headObject(HeadObjectRequest.builder()
    .bucket(bucket)
    .key(key)
    .build());

最佳实践

1. 使用路径风格访问

1
2

.builder()
.forcePathStyle(true) // 关键配置

2. 文件键设计

使用日期路径便于清理和管理
添加 UUID 防止文件名冲突
中文文件名转拼音或 MD5

3. 异常处理

始终处理 NoSuchBucketException 和 NoSuchKeyException
对 IO 操作进行 try-catch
记录详细日志便于排查

4. 性能优化

使用流式上传避免大文件内存溢出
合理设置 contentLength
使用连接池（AWS SDK 默认使用连接池）

总结

RustFS 通过 S3 兼容 API 为我们提供了简单高效的对象存储能力。结合 AWS SDK v2，我们可以：

轻松上传、下载、删除文件
设计清晰的文件键结构
处理中文文件名
进行完整的错误处理

本指南覆盖了 RustFS 的基础使用、进阶操作和最佳实践，希望对你的项目开发有所帮助。

S3 on Zell Blog

RustFS使用指南

什么是 RustFS？

项目集成

Maven 依赖

S3 客户端配置

配置文件

基础使用

文件上传

文件下载

文件删除

文件是否存在检查

获取文件信息

文件键设计

按日期和 UUID 组织文件

中文文件名处理

存储桶操作

确保存储桶存在

列出存储桶中的对象

完整服务封装

错误处理

常见错误处理

AWS SDK v2 常用操作一览

上传下载

存储桶操作

最佳实践

1. 使用路径风格访问

2. 文件键设计

3. 异常处理

4. 性能优化

总结