目录

1、添加相关配置

2、添加依赖

3、实现方法

1️⃣基础版:

2️⃣优化版(推荐使用):

3️⃣上传远程主机上的目录内容:

4️⃣直接上传远程主机中的目录内容


业务背景:需要需要minio进行上传指定目录下所有的内容,具体如下:

1、添加相关配置


# minio 服务
minio:#Minio服务所在地址endpoint: http://192.168.1.20:9000# 默认存储桶名称 bucketName: minio-upload#访问的keyaccessKey: zItdfkAbUVWNkIo#访问的秘钥secretKey: JlZp3WTG6PFEckFW7UHXqXSzGu1ICwMXQ6Ep#存储桶列表名称bucketNames: data-upload,testdata

配置文件 MinioConfig


import io.minio.MinioClient;
import io.minio.errors.ServerException;
import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;import java.util.List;/*** minio 配置** @author ATB* @date 2025*/
@Data
@Configuration
@ConfigurationProperties(prefix = "minio")
public class MinioConfig {/*** 端点*/private String endpoint;/*** 访问密钥*/private String accessKey;/*** 密钥*/private String secretKey;/*** 存储桶名称*/private String bucketName;/*** 存储桶名称列表*/private List<String> bucketNames;/*** 注入 Minio 客户端** @return {@link MinioClient }*/@Beanpublic MinioClient minioClient() throws ServerException {try {return MinioClient.builder().endpoint(endpoint).credentials(accessKey, secretKey).build();} catch (Exception e) {throw new ServerException("-----创建Minio客户端失败-----" + e.getMessage() , 202, "未处理");}//return MinioClient.builder()//    .endpoint(endpoint)//    .credentials(accessKey, secretKey)//    .build();}
}

当然也可以不用这个配置类的,需要使用@Value获取配置参数就行。 

2、添加依赖

        <!-- https://mvnrepository.com/artifact/io.minio/minio --><dependency><groupId>io.minio</groupId><artifactId>minio</artifactId><version>8.5.17</version></dependency><dependency><groupId>com.amazonaws</groupId><artifactId>aws-java-sdk-s3</artifactId><version>1.12.540</version></dependency>

3、实现方法

1️⃣和2️⃣是对本地的目录进行操作,3️⃣是对远程机器的目录进行操作。

1️⃣基础版:

其中minioConfig.getEndpoint()就是什么配置类中的参数,也可以使用如下方法获取配置文件的内容,还可以直接写成固定值都是可以的。

@Value("minio.endpoint")
private String endpoint;
    @Autowiredprivate MinioConfig minioConfig;@Resourceprivate MinioClient minioClient;/*** 上传目录** @param bucketName    存储桶名称* @param directoryPath directory path (目录路径) /home/data/app/models* @return {@link ResponseEntity }<{@link Object }>*/@ApiOperation(value = "上传目录")@GetMapping("/uploadCatalog")public ResponseEntity<Object> uploadCatalog(@RequestParam("bucketName") String bucketName, @RequestParam("directoryPath") String directoryPath) {System.out.println("bucketName = " + bucketName);System.out.println("directoryPath = " + directoryPath);try {// 初始化 MinIO 客户端MinioClient minioClient = MinioClient.builder().endpoint(minioConfig.getEndpoint()).credentials(minioConfig.getAccessKey(), minioConfig.getSecretKey()).build();// 递归遍历目录及其子目录Files.walkFileTree(Paths.get(directoryPath), new SimpleFileVisitor<Path>() {@Overridepublic FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {// 将子目录视为普通文件上传String relativePath;if (directoryPath.length() + 1 > dir.toString().length()) {// 如果超出范围,可以选择返回空字符串或其他默认值relativePath = " /home/data/app/models";} else {relativePath = StringUtils.substringAfter(dir.toString(), directoryPath);}log.info("Uploading directory: " + relativePath);try {// 上传目录作为文件minioClient.putObject(PutObjectArgs.builder().bucket(bucketName)// 结尾加斜杠表示目录.object("/model-file-path/" + relativePath).stream(new FileInputStream(dir.toString()), 0, -1)// 设置为目录类型.contentType("application/x-directory").build());log.info("Uploaded directory: " + relativePath);} catch (Exception e) {log.error("Failed to upload directory: " + dir.getFileName());e.printStackTrace();}// 继续遍历子目录return FileVisitResult.CONTINUE;}@Overridepublic FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {// 上传文件String relativePath = file.toString().substring(directoryPath.length() + 1);log.info("Uploading file: " + relativePath);try {minioClient.putObject(PutObjectArgs.builder().bucket(bucketName).object("/model-file-path/" + relativePath).stream(new FileInputStream(file.toFile()), Files.size(file), -1)// 自动检测文件类型.contentType(Files.probeContentType(file)) .build());log.info("Uploaded file: " + relativePath);} catch (Exception e) {log.warn("Failed to upload file: " + file.getFileName());e.printStackTrace();}// 继续遍历下一个文件return FileVisitResult.CONTINUE;}@Overridepublic FileVisitResult visitFileFailed(Path file, IOException exc) throws IOException {log.warn("Failed to process file: " + file.getFileName() + ", reason: " + exc.getMessage());// 忽略失败的文件return FileVisitResult.CONTINUE;}});log.info("All files and directories uploaded successfully.");} catch (IOException e) {log.warn("Error occurred: " + e.getMessage());e.printStackTrace();}HashMap<String, Object> response = new HashMap<>();response.put("data", "directoryPath");response.put("code", HttpStatus.OK.value());response.put("msg", "上传成功");return ResponseEntity.status(HttpStatus.OK).body(response);}

2️⃣优化版(推荐使用):

不同的文件类型也都可以直接上传

/*** 上传内容目录** @param bucketName    存储桶名称* @param directoryPath directory path (目录路径)* @return {@link ResponseEntity }<{@link Object }>*/@GetMapping("/uploadContentsCatalog")public ResponseEntity<Object> uploadContentsCatalog(@RequestParam("bucketName") String bucketName,@RequestParam("directoryPath") String directoryPath) {log.info("Upload catalog: bucket={}, path={}", bucketName, directoryPath);Map<String, Object> response = new HashMap<>();List<String> successList = new ArrayList<>();List<String> failedList = new ArrayList<>();try {MinioClient minioClient = MinioClient.builder().endpoint(minioConfig.getEndpoint()).credentials(minioConfig.getAccessKey(), minioConfig.getSecretKey()).build();Path basePath = Paths.get(directoryPath);Files.walkFileTree(basePath, new SimpleFileVisitor<Path>() {@Overridepublic FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) {if (!dir.equals(basePath)) { // 根目录不需要作为“目录对象”上传String relativePath = basePath.relativize(dir).toString().replace("\\", "/") + "/";try {// “模拟”上传一个空对象表示目录(非必要,可跳过)minioClient.putObject(PutObjectArgs.builder().bucket(bucketName).object("/model-file-path/" + relativePath).stream(new ByteArrayInputStream(new byte[0]), 0, -1).contentType("application/x-directory").build());log.info("Uploaded directory: {}", relativePath);successList.add(relativePath);} catch (Exception e) {log.error("Failed to upload directory: {}", relativePath, e);failedList.add(relativePath);}}return FileVisitResult.CONTINUE;}@Overridepublic FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {String relativePath = basePath.relativize(file).toString().replace("\\", "/");try (InputStream is = new FileInputStream(file.toFile())) {minioClient.putObject(PutObjectArgs.builder().bucket(bucketName).object("/model-file-path/" + relativePath).stream(is, Files.size(file), -1)//.contentType(Files.probeContentType(file)).contentType(Optional.ofNullable(Files.probeContentType(file)).orElse("application/octet-stream")).build());log.info("Uploaded file: {}", relativePath);successList.add(relativePath);} catch (Exception e) {log.warn("Failed to upload file: {}", relativePath, e);failedList.add(relativePath);}return FileVisitResult.CONTINUE;}@Overridepublic FileVisitResult visitFileFailed(Path file, IOException exc) {String relativePath = basePath.relativize(file).toString().replace("\\", "/");log.warn("Could not access file: {}, reason: {}", relativePath, exc.getMessage());failedList.add(relativePath);return FileVisitResult.CONTINUE;}});} catch (Exception e) {log.error("Unexpected error during directory upload", e);return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).body(Map.of("code", 500, "msg", "上传过程中出现错误", "error", e.getMessage()));}response.put("code", HttpStatus.OK.value());response.put("msg", "上传完成");response.put("successCount", successList.size());response.put("failCount", failedList.size());response.put("successItems", successList);response.put("failedItems", failedList);return ResponseEntity.ok(response);}

3️⃣上传远程主机上的目录内容:

实现从远程主机上传目录到 MinIO,你可以通过 SSH(SFTP 或 SCP)下载远程目录到本地临时目录,然后按之前逻辑上传至 MinIO。

添加依赖(JSch 连接远程机器)

<dependency><groupId>com.jcraft</groupId><artifactId>jsch</artifactId><version>0.1.55</version>
</dependency>

具体实现

    /*** 上传远程目录** @param bucketName      存储桶名称* @param remoteDirectory remote 目录* @return {@link ResponseEntity }<{@link Object }>*/@PostMapping("/uploadRemoteCatalog")public ResponseEntity<Object> uploadRemoteCatalog(@RequestParam("bucketName") String bucketName,@RequestParam("remoteDirectory") String remoteDirectory) {String remoteHost = "192.168.1.88";String username = "root";String password = "12345678";String tempDir = System.getProperty("java.io.tmpdir") + "/upload_" + UUID.randomUUID();File localTempDir = new File(tempDir);localTempDir.mkdirs();log.info("Start downloading remote dir [{}] from {} to local temp: {}", remoteDirectory, remoteHost, tempDir);Map<String, Object> response = new HashMap<>();List<String> successList = new ArrayList<>();List<String> failedList = new ArrayList<>();try {// 1. 下载远程目录到本地临时目录JSch jsch = new JSch();Session session = jsch.getSession(username, remoteHost, 22);session.setPassword(password);session.setConfig("StrictHostKeyChecking", "no");session.connect();Channel channel = session.openChannel("sftp");channel.connect();ChannelSftp sftpChannel = (ChannelSftp) channel;// 下载远程目录到本地临时目录downloadRemoteDirectory(sftpChannel, remoteDirectory, tempDir);sftpChannel.exit();session.disconnect();log.info("Remote directory downloaded to local temp: {}", tempDir);// 2. 上传本地目录到 MinIO(与之前逻辑一致)Path basePath = Paths.get(tempDir);MinioClient minioClient = MinioClient.builder().endpoint(minioConfig.getEndpoint()).credentials(minioConfig.getAccessKey(), minioConfig.getSecretKey()).build();Files.walkFileTree(basePath, new SimpleFileVisitor<Path>() {@Overridepublic FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) {if (!dir.equals(basePath)) {String relativePath = basePath.relativize(dir).toString().replace("\\", "/") + "/";try {minioClient.putObject(PutObjectArgs.builder().bucket(bucketName).object("/model-file-path/" + relativePath).stream(new ByteArrayInputStream(new byte[0]), 0, -1).contentType("application/x-directory").build());successList.add(relativePath);} catch (Exception e) {log.error("Failed to upload directory: {}", relativePath, e);failedList.add(relativePath);}}return FileVisitResult.CONTINUE;}@Overridepublic FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {String relativePath = basePath.relativize(file).toString().replace("\\", "/");try (InputStream is = new FileInputStream(file.toFile())) {String contentType = Optional.ofNullable(Files.probeContentType(file)).orElse("application/octet-stream");minioClient.putObject(PutObjectArgs.builder().bucket(bucketName).object("/model-file-path/" + relativePath).stream(is, Files.size(file), -1).contentType(contentType).build());successList.add(relativePath);} catch (Exception e) {log.warn("Failed to upload file: {}", relativePath, e);failedList.add(relativePath);}return FileVisitResult.CONTINUE;}});} catch (Exception e) {log.error("Upload from remote server failed", e);return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).body(Map.of("code", 500, "msg", "远程上传失败", "error", e.getMessage()));} finally {// 3. 清理删除临时目录try {log.warn("Delete temp dir: {}", localTempDir);FileUtils.deleteDirectory(localTempDir);} catch (IOException e) {log.warn("Failed to delete temp dir: {}", tempDir);}}response.put("code", HttpStatus.OK.value());response.put("msg", "上传完成");response.put("successCount", successList.size());response.put("failCount", failedList.size());response.put("successItems", successList);response.put("failedItems", failedList);return ResponseEntity.ok(response);}/*** 下载 Remote Directory** @param sftp      SFTP* @param remoteDir 远程目录* @param localDir  本地目录* @throws SftpException SFTP 异常*/private void downloadRemoteDirectory(ChannelSftp sftp, String remoteDir, String localDir) throws SftpException {File localFolder = new File(localDir);if (!localFolder.exists()) {localFolder.mkdirs();}@SuppressWarnings("unchecked")Vector<ChannelSftp.LsEntry> entries = sftp.ls(remoteDir);for (ChannelSftp.LsEntry entry : entries) {String fileName = entry.getFilename();if (".".equals(fileName) || "..".equals(fileName)) continue;String remotePath = remoteDir + "/" + fileName;String localPath = localDir + "/" + fileName;if (entry.getAttrs().isDir()) {downloadRemoteDirectory(sftp, remotePath, localPath);} else {try (OutputStream output = new FileOutputStream(localPath)) {sftp.get(remotePath, output);} catch (IOException e) {log.error("Failed to download file: {}", remotePath, e);}}}}

至此可以实现远程上传到minio中。远程上传文件目录还可以继续优化,如考虑:支持私钥登录、上传前压缩、并发、上传进度等,所以可以形成一个远程上传的工具类,具体如下:


import com.jcraft.jsch.*;
import io.minio.MinioClient;
import io.minio.PutObjectArgs;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.io.FileUtils;import java.io.*;
import java.nio.file.*;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.*;/*** 远程上传器** @author ATB* @date 2025*/
@Slf4j
public class RemoteUploader {public static class SSHConfig {public String host;public int port = 22;public String username;public String password;public SSHConfig(String host, String username, String password) {this.host = host;this.username = username;this.password = password;}}public static class UploadResult {public List<String> success = new ArrayList<>();public List<String> failed = new ArrayList<>();}/*** 1️⃣ 直接上传** @param sshConfig   SSH 配置* @param remoteDir   远程目录* @param bucketName  存储桶名称* @param minioClient Minio 客户端* @return {@link UploadResult }* @throws Exception 例外*/public static UploadResult uploadRemoteDirectoryToMinio(SSHConfig sshConfig,String remoteDir,String bucketName,MinioClient minioClient) throws Exception {// 1. 下载远程目录String tempDirPath = System.getProperty("java.io.tmpdir") + "/upload_" + UUID.randomUUID();File tempDir = new File(tempDirPath);tempDir.mkdirs();JSch jsch = new JSch();Session session = jsch.getSession(sshConfig.username, sshConfig.host, sshConfig.port);session.setPassword(sshConfig.password);session.setConfig("StrictHostKeyChecking", "no");session.connect();ChannelSftp sftp = (ChannelSftp) session.openChannel("sftp");sftp.connect();downloadRemoteDirectory(sftp, remoteDir, tempDirPath);sftp.disconnect();session.disconnect();// 2. 上传目录到 MinIOUploadResult result = uploadLocalDirectoryToMinio(tempDirPath, bucketName, minioClient);// 3. 删除临时目录try {FileUtils.deleteDirectory(tempDir);} catch (IOException e) {log.warn("Failed to delete temp dir: {}", tempDirPath);}return result;}private static void downloadRemoteDirectory(ChannelSftp sftp, String remoteDir, String localDir) throws SftpException {new File(localDir).mkdirs();@SuppressWarnings("unchecked")Vector<ChannelSftp.LsEntry> files = sftp.ls(remoteDir);for (ChannelSftp.LsEntry entry : files) {String name = entry.getFilename();if (".".equals(name) || "..".equals(name)) continue;String remotePath = remoteDir + "/" + name;String localPath = localDir + "/" + name;if (entry.getAttrs().isDir()) {downloadRemoteDirectory(sftp, remotePath, localPath);} else {try (OutputStream os = new FileOutputStream(localPath)) {sftp.get(remotePath, os);} catch (IOException e) {log.error("Failed to download file: {}", remotePath, e);}}}}/*** 2️⃣ 将远程目录上传到压缩后 Minio** @param sshConfig   SSH 配置* @param remoteDir   远程目录* @param bucketName  存储桶名称* @param minioClient Minio 客户端* @return {@link UploadResult }* @throws Exception 例外*/public static UploadResult uploadRemoteDirectoryToMinioWithCompression(SSHConfig sshConfig,String remoteDir,String bucketName,MinioClient minioClient) throws Exception {// 建立 SSH 会话JSch jsch = new JSch();Session session = jsch.getSession(sshConfig.username, sshConfig.host, sshConfig.port);session.setPassword(sshConfig.password);session.setConfig("StrictHostKeyChecking", "no");session.connect();// 打包远程目录String remoteTarPath = remoteTarDirectory(session, remoteDir);// SFTP 下载 tar.gzChannelSftp sftp = (ChannelSftp) session.openChannel("sftp");sftp.connect();String localTarPath = downloadRemoteTar(sftp, remoteTarPath);// 可选:清理远程文件sftp.rm(remoteTarPath);sftp.disconnect();session.disconnect();// 本地解压String extractPath = extractTarGz(localTarPath);// 上传解压后的目录UploadResult result = uploadLocalDirectoryToMinio(extractPath, bucketName, minioClient);// 清理临时文件new File(localTarPath).delete();FileUtils.deleteDirectory(new File(extractPath));return result;}/*** 在远程打包 .tar.gz** @param session   会期* @param remoteDir 远程目录* @return {@link String }* @throws JSchException JSCH 例外* @throws IOException   io异常*/private static String remoteTarDirectory(Session session, String remoteDir) throws JSchException, IOException {String parent = remoteDir.endsWith("/") ? remoteDir.substring(0, remoteDir.length() - 1) : remoteDir;String tarName = "/tmp/upload_" + UUID.randomUUID() + ".tar.gz";String command = String.format("tar -czf %s -C %s .", tarName, parent);log.info("Executing remote tar: {}", command);ChannelExec exec = (ChannelExec) session.openChannel("exec");exec.setCommand(command);exec.setInputStream(null);exec.setErrStream(System.err);exec.connect();InputStream input = exec.getInputStream();byte[] tmp = new byte[1024];while (input.read(tmp, 0, 1024) >= 0) {}exec.disconnect();return tarName;}/*** 下载 .tar.gz 文件** @param sftp          SFTP* @param remoteTarPath 远程 tar 路径* @return {@link String }* @throws Exception 例外*/private static String downloadRemoteTar(ChannelSftp sftp, String remoteTarPath) throws Exception {String localTarPath = System.getProperty("java.io.tmpdir") + "/" + UUID.randomUUID() + ".tar.gz";try (FileOutputStream fos = new FileOutputStream(localTarPath)) {sftp.get(remoteTarPath, fos);}return localTarPath;}/*** 解压 .tar.gz** @param tarGzPath tar gz 路径* @return {@link String }* @throws IOException io异常*/private static String extractTarGz(String tarGzPath) throws IOException {String outputDir = tarGzPath.replace(".tar.gz", "_extracted");Files.createDirectories(Paths.get(outputDir));ProcessBuilder pb = new ProcessBuilder("tar", "-xzf", tarGzPath, "-C", outputDir);Process process = pb.start();try {if (process.waitFor() != 0) throw new RuntimeException("Failed to extract tar.gz");} catch (InterruptedException e) {Thread.currentThread().interrupt();}return outputDir;}/*** 将本地目录上传到 Minio** @param localDirPath 本地目录路径* @param bucketName   存储桶名称* @param minioClient  Minio 客户端* @return {@link UploadResult }* @throws IOException io异常*/private static UploadResult uploadLocalDirectoryToMinio(String localDirPath,String bucketName,MinioClient minioClient) throws IOException {UploadResult result = new UploadResult();Path basePath = Paths.get(localDirPath);Files.walkFileTree(basePath, new SimpleFileVisitor<Path>() {@Overridepublic FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) {if (!dir.equals(basePath)) {String relativePath = basePath.relativize(dir).toString().replace("\\", "/") + "/";try {minioClient.putObject(PutObjectArgs.builder().bucket(bucketName).object(relativePath).stream(new ByteArrayInputStream(new byte[0]), 0, -1).contentType("application/x-directory").build());result.success.add(relativePath);} catch (Exception e) {log.warn("Failed to upload directory: {}", relativePath, e);result.failed.add(relativePath);}}return FileVisitResult.CONTINUE;}@Overridepublic FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {String relativePath = basePath.relativize(file).toString().replace("\\", "/");try (InputStream is = new FileInputStream(file.toFile())) {String contentType = Optional.ofNullable(Files.probeContentType(file)).orElse("application/octet-stream");minioClient.putObject(PutObjectArgs.builder().bucket(bucketName).object(relativePath).stream(is, Files.size(file), -1).contentType(contentType).build());result.success.add(relativePath);} catch (Exception e) {log.warn("Failed to upload file: {}", relativePath, e);result.failed.add(relativePath);}return FileVisitResult.CONTINUE;}});return result;}//==[考虑直接远程上传等方式]================================================================================================================// TODO: 考虑直接远程上传等方式public void uploadRemoteFileToMinio(SSHConfig sshConfig, String remoteFilePath, String bucketName, String objectName, MinioClient minioClient) throws Exception {// 建立 SSH 会话JSch jsch = new JSch();Session session = jsch.getSession(sshConfig.username, sshConfig.host, sshConfig.port);session.setPassword(sshConfig.password);session.setConfig("StrictHostKeyChecking", "no");session.connect();ChannelSftp sftp = (ChannelSftp) session.openChannel("sftp");sftp.connect();try (InputStream remoteStream = sftp.get(remoteFilePath)) {minioClient.putObject(PutObjectArgs.builder().bucket(bucketName).object(objectName).stream(remoteStream, -1, PutObjectArgs.MIN_MULTIPART_SIZE)// 可通过文件后缀判断更具体类型.contentType("application/octet-stream").build());log.info("Uploaded from remote: {} to MinIO: {}", remoteFilePath, objectName);} finally {sftp.disconnect();}}/*** 从远程主机读取文件流,上传到 MinIO** @param sshHost     远程主机 IP 或域名* @param sshPort     SSH 端口,默认 22* @param username    用户名* @param password    密码(或设置为 null 以使用私钥登录)* @param privateKey  私钥路径(或 null)* @param remotePath  远程文件绝对路径* @param minioClient MinIO 客户端* @param bucketName  MinIO bucket 名称* @param objectName  上传后对象名(key)*/public static void uploadRemoteFileToMinio(String sshHost,int sshPort,String username,String password,String privateKey,String remotePath,MinioClient minioClient,String bucketName,String objectName) throws Exception {JSch jsch = new JSch();if (privateKey != null && !privateKey.isBlank()) {jsch.addIdentity(privateKey);}Session session = jsch.getSession(username, sshHost, sshPort);if (password != null && !password.isBlank()) {session.setPassword(password);}session.setConfig("StrictHostKeyChecking", "no");session.connect();ChannelSftp sftp = (ChannelSftp) session.openChannel("sftp");sftp.connect();try (InputStream inputStream = sftp.get(remotePath)) {long fileSize = sftp.lstat(remotePath).getSize();minioClient.putObject(PutObjectArgs.builder().bucket(bucketName).object(objectName).stream(inputStream, fileSize, -1).contentType("application/octet-stream").build());log.info("上传成功: remote={}, object={}", remotePath, objectName);} catch (Exception e) {log.error("上传失败: {}", e.getMessage(), e);throw e;} finally {sftp.disconnect();session.disconnect();}}}

实现调用:

/*** 上传压缩** @param bucketName 存储桶名称* @param remotePath 远程路径* @return {@link ResponseEntity }<{@link ? }>*/@PostMapping("/uploadCompressed")public ResponseEntity<?> uploadCompressed(@RequestParam String bucketName,@RequestParam String remotePath) {try {RemoteUploader.SSHConfig ssh = new RemoteUploader.SSHConfig("192.168.1.88", "root", "12345678");MinioClient client = MinioClient.builder().endpoint(minioConfig.getEndpoint()).credentials(minioConfig.getAccessKey(), minioConfig.getSecretKey()).build();// 直接远程上传// RemoteUploader.UploadResult result = RemoteUploader.uploadRemoteDirectoryToMinio(ssh, remotePath, bucketName, client);// 压缩传输上传RemoteUploader.UploadResult result = RemoteUploader.uploadRemoteDirectoryToMinioWithCompression(ssh, remotePath, bucketName, client);return ResponseEntity.ok(Map.of("code", 200,"msg", "上传完成(压缩)","successCount", result.success.size(),"failCount", result.failed.size(),"successItems", result.success,"failedItems", result.failed));} catch (Exception e) {return ResponseEntity.status(500).body(Map.of("code", 500,"msg", "上传失败","error", e.getMessage()));}}

4️⃣直接上传远程主机中的目录内容

添加工具类:


import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.Session;
import com.jcraft.jsch.SftpException;
import io.minio.MinioClient;
import io.minio.PutObjectArgs;
import lombok.extern.slf4j.Slf4j;import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
import java.util.Vector;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;/*** remote dir uploader** @author ATB* &date  2025/05/28*/
@Slf4j
public class RemoteDirUploader {public static class UploadResult {public List<String> success = new ArrayList<>();public List<String> failed = new ArrayList<>();}/*** 将远程目录上传到 Minio** @param sshHost     SSH 主机* @param sshPort     SSH 端口* @param username    用户名* @param password    密码* @param privateKey  私钥* @param remoteDir   远程目录* @param bucketName  存储桶名称* @param minioPrefix minio 前缀* @param minioClient Minio 客户端* @throws Exception 例外*/public static UploadResult uploadRemoteDirectoryToMinio(String sshHost, int sshPort,String username, String password, String privateKey,String remoteDir, String bucketName, String minioPrefix,MinioClient minioClient) throws Exception {JSch jsch = new JSch();if (privateKey != null && !privateKey.isBlank()) {jsch.addIdentity(privateKey);}Session session = jsch.getSession(username, sshHost, sshPort);if (password != null && !password.isBlank()) {session.setPassword(password);}session.setConfig("StrictHostKeyChecking", "no");session.connect();ChannelSftp sftp = (ChannelSftp) session.openChannel("sftp");sftp.connect();UploadResult result;try {result = uploadRecursive(sftp, remoteDir, remoteDir, bucketName, minioPrefix, minioClient);} finally {sftp.disconnect();session.disconnect();}return result;}private static UploadResult uploadRecursive(ChannelSftp sftp, String basePath, String currentPath,String bucketName, String minioPrefix,MinioClient minioClient) throws Exception {UploadResult result = new UploadResult();@SuppressWarnings("unchecked")Vector<ChannelSftp.LsEntry> entries = sftp.ls(currentPath);for (ChannelSftp.LsEntry entry : entries) {String filename = entry.getFilename();if (filename.equals(".") || filename.equals("..")) continue;String fullPath = currentPath + "/" + filename;String relativePath = fullPath.substring(basePath.length()).replaceFirst("^/", "");String objectName = minioPrefix + relativePath;if (entry.getAttrs().isDir()) {// 递归子目录uploadRecursive(sftp, basePath, fullPath, bucketName, minioPrefix, minioClient);} else {long fileSize = entry.getAttrs().getSize();try (InputStream inputStream = sftp.get(fullPath)) {minioClient.putObject(PutObjectArgs.builder().bucket(bucketName).object(objectName).stream(inputStream, fileSize, -1).contentType("application/octet-stream").build());log.info("上传成功: {}", objectName);result.success.add(objectName);} catch (Exception e) {log.warn("上传失败: {} - {}", objectName, e.getMessage());result.failed.add(objectName);}}}return result;}
}

然后接口调用:

/*** 流式直接上传** @param bucketName 存储桶名称* @param remotePath 远程路径* @return {@link ResponseEntity }<{@link ? }>*/@PostMapping("/streamingUploads")public ResponseEntity<?> streamingUploads(@RequestParam String bucketName,@RequestParam String remotePath) {try {MinioClient client = MinioClient.builder().endpoint(minioConfig.getEndpoint()).credentials(minioConfig.getAccessKey(), minioConfig.getSecretKey()).build();RemoteDirUploader.UploadResult result = RemoteDirUploader.uploadRemoteDirectoryToMinio("192.168.5.88", 22, "root", "123456", null,remotePath, bucketName, "model-file/data/",minioClient);return ResponseEntity.ok(Map.of("code", 200,"msg", "上传完成","successCount", result.success.size(),"failCount", result.failed.size(),"successItems", result.success,"failedItems", result.failed));} catch (Exception e) {return ResponseEntity.status(500).body(Map.of("code", 500,"msg", "上传失败","error", e.getMessage()));}}

至此!就可以实现本地或者是远程机器上的目录上传到minio中了!

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。
如若转载,请注明出处:http://www.pswp.cn/bicheng/82694.shtml
繁体地址,请注明出处:http://hk.pswp.cn/bicheng/82694.shtml
英文地址,请注明出处:http://en.pswp.cn/bicheng/82694.shtml

如若内容造成侵权/违法违规/事实不符,请联系英文站点网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

Python的分布式网络爬虫系统实现

1. 系统架构概述 一个典型的分布式网络爬虫系统通常包含以下几个核心组件&#xff1a; 1.主节点&#xff08;Master Node&#xff09;&#xff1a; 任务调度&#xff1a;负责将抓取任务分配给各个工作节点。URL 管理&#xff1a;维护待抓取的 URL 队列和已抓取的 URL 集合&a…

AI工具的选择:Dify还是传统工具?

从纯技术视角出发&#xff0c;选择Dify还是传统开发工具需要基于六个核心维度进行理性决策。以下为结构化分析框架&#xff0c;附典型场景示例&#xff1a; 1. 开发效率 vs 控制力权衡矩阵 维度Dify优势场景传统工具优势场景迭代速度需求明确的标准CRUD&#xff08;如后台管理…

2.3 TypeScript 非空断言操作符(后缀 !)详解

在 TypeScript 中&#xff0c;当你开启了严格的空值检查&#xff08;strictNullChecks&#xff09;后&#xff0c;变量如果可能是 null 或 undefined&#xff0c;就必须在使用前进行显式的判断。为了在某些场景下简化代码&#xff0c;TypeScript 提供了非空断言操作符&#xff…

深度学习:损失函数与激活函数全解析

目录 深度学习中常见的损失函数和激活函数详解引言一、损失函数详解1.1 损失函数的作用与分类1.2 回归任务损失函数1.2.1 均方误差&#xff08;MSE&#xff09;1.2.2 平均绝对误差&#xff08;MAE&#xff09; 1.3 分类任务损失函数1.3.1 交叉熵损失&#xff08;Cross-Entropy&…

掌握 npm 核心操作:从安装到管理依赖的完整指南

图为开发者正在终端操作npm命令&#xff0c;图片来源&#xff1a;Unsplash 作为 Node.js 生态的基石&#xff0c;npm&#xff08;Node Package Manager&#xff09;是每位开发者必须精通的工具。每天有超过 1700 万个项目通过 npm 共享代码&#xff0c;其重要性不言而喻。本文…

Elasticsearch的运维

Elasticsearch 运维工作详解&#xff1a;从基础保障到性能优化 Elasticsearch&#xff08;简称 ES&#xff09;作为分布式搜索和分析引擎&#xff0c;其运维工作需要兼顾集群稳定性、性能效率及数据安全。以下从核心运维模块展开说明&#xff0c;结合实践场景提供可落地的方案…

国产三维CAD皇冠CAD(CrownCAD)建模教程:汽车电池

在线解读『汽车电池』的三维建模流程&#xff0c;讲解3D草图、保存实体、拉伸凸台/基体、设置外观等操作技巧&#xff0c;一起和皇冠CAD&#xff08;CrownCAD&#xff09;学习制作步骤吧&#xff01; 汽车电池&#xff08;通常指铅酸蓄电池或锂离子电池&#xff09;是车辆电气系…

深入理解 JDK、JRE 和 JVM 的区别

在 Java 中&#xff0c;JDK、JRE 和 JVM 是非常重要的概念&#xff0c;它们各自扮演着不同的角色&#xff0c;却又紧密相连。今天&#xff0c;就让我们来详细探讨一下它们之间的区别。 一、JVM JVM 即 Java 虚拟机&#xff0c;它是整个 Java 技术体系的核心。JVM 提供了 Java…

云电脑显卡性能终极对决:ToDesk云电脑/顺网云/海马云,谁才是4K游戏之王?

一、引言 1.1 云电脑的算力革命 云电脑与传统PC的算力供给差异 传统PC的算力构建依赖用户一次性配置本地硬件&#xff0c;特别是CPU与显卡&#xff08;GPU&#xff09;。而在高性能计算和游戏图形渲染等任务中&#xff0c;GPU的能力往往成为决定体验上限的核心因素。随着游戏分…

撤销Conda初始化

在安装miniconda3的过程中&#xff0c;最后系统会出现这一行提示用户可以选择自动初始化&#xff0c;这样的话&#xff0c;系统每次启动就会自动启动基础&#xff08;base&#xff09;环境。 但是我们也可以通过 conda init --reverse $shell 来撤销 Conda 的初始化设置。这将恢…

Flask-SQLAlchemy数据库查询:query

1、为什么可以用 模型类.query 来查询数据库&#xff1f; 在 Flask 中使用 SQLAlchemy ORM 时&#xff0c;所有继承自 db.Model 的模型类都会自动获得一个 query 属性。 其本质是 db.session.query(模型类) 的快捷方式&#xff0c;无需显式操作 db.session。 代码示例&#…

【免费】【无需登录/关注】度分秒转换在线工具

UVE Toolbox 功能概述 这是一个用于地理坐标转换的在线工具&#xff0c;支持两种转换模式&#xff1a; 十进制度 → 度分秒 度分秒 → 十进制度 使用方法 十进制度转度分秒 在"经度"输入框中输入十进制度格式的经度值&#xff08;例如&#xff1a;121.46694&am…

怎么判断一个Android APP使用了React Native 这个跨端框架

要判断一个 Android 应用是否使用了 React Native 框架&#xff0c;可以通过以下方法逐步验证&#xff1a; 一、安装包结构分析 1. 解压 APK 将 .apk 文件重命名为 .zip 并解压&#xff0c;检查以下特征文件&#xff1a; • assets/index.android.bundle&#xff1a; React Na…

Pluto实验报告——基于2ASK的简易的通信系统

一、实验目的 1. 熟悉并掌握PLUTO SDR 主动学习模块的使用&#xff1b; 2.通过matlab 编码与adalm pluto 相配合达成一个简易的通信系统&#xff0c;并能 够传输一些较为简单的信息。 二、实验原理 2ASK 调制原理&#xff1a; 振幅键控是指利用载波的振幅变化来传递数字基带信…

Ubuntu 24-部署FTP和自定义用户

目录 一、 安装 vsftpd 二、创建 FTP 数据目录 三、创建 FTP 用户 四、配置 vsftpd 五、重启 vsftpd 服务 六、增加新用户脚本 一、 安装 vsftpd sudo apt update sudo apt install vsftpd -y 二、创建 FTP 数据目录 sudo mkdir -p /data/ftp sudo chown nobody:nogrou…

MySQL问题:什么是MySQL的中的最左匹配原则?

是指在复合索引中&#xff0c;查询条件需要按照索引列的顺序从最左侧列开始依次匹配。只有查询条件中的列按照索引的最左边列开始进行匹配&#xff0c;索引才能被有效使用&#xff0c;但有时虽然不是正常顺序&#xff0c;由于MySQL中存在优化器&#xff0c;会自动调整顺序&…

2025软考软件设计师题目

选择题&#xff08;综合题&#xff09; 确定得分的 1、Linux外设目录是什么 /dev。存储磁盘的目录 2、Linux外设sdc类型设备属于什么 scsi hard disk。根据第一个字母s盲猜的 3、计算机中让程序计数器PC不能指向当前运行程序的技术是 流水线。根据流水线的原理 4、Python程…

Deep Evidential Regression

摘要 翻译&#xff1a; 确定性神经网络&#xff08;NNs&#xff09;正日益部署在安全关键领域&#xff0c;其中校准良好、鲁棒且高效的不确定性度量至关重要。本文提出一种新颖方法&#xff0c;用于训练非贝叶斯神经网络以同时估计连续目标值及其关联证据&#xff0c;从而学习…

每天掌握一个Linux命令 - sqlite3

Linux 命令工具 sqlite3 使用指南 一、工具概述 sqlite3 是 SQLite 数据库的命令行工具&#xff0c;用于在 Linux 系统中直接操作 SQLite 数据库&#xff08;轻量级、无服务器、嵌入式关系型数据库&#xff09;。 核心特点&#xff1a; 无需安装数据库服务&#xff0c;直接通…

leetcode:2160. 拆分数位后四位数字的最小和(python3解法,数学相关算法题)

难度&#xff1a;简单 给你一个四位 正 整数 num 。请你使用 num 中的 数位 &#xff0c;将 num 拆成两个新的整数 new1 和 new2 。new1 和 new2 中可以有 前导 0 &#xff0c;且 num 中 所有 数位都必须使用。 比方说&#xff0c;给你 num 2932 &#xff0c;你拥有的数位包括…