AWS SDK for JAVA V2 でS3にMultipart Uploadする

検索してもAWSのサンプルコード以外にヒットしないので需要があるのかわからないけどコードを書いたので記事にします。 localstack で確認していますが、AWSでは動かしていません 🙃

テスト環境

Library	Version
Kotlin	1.4.21
Spring Boot	2.4.2
AWS SDK for JAVA V2	2.15.38

コード

fun uploadByMultipart(file: Path) {

    // s3 client for localstack
    // aws環境ではendpointOverrideは不要
    val s3 = S3Client.builder()
        .region(DefaultAwsRegionProviderChain().region)
        .endpointOverride(URI.create(ENDPOINT_URL))
        .build()

    val size = file.toFile().length()
    val key = file.fileName.toString()

    if (size < MIN_MULTIPART_SIZE) {
        throw IllegalArgumentException()
    }

    // 分割してファイルを読み込めるようにRandomAccessFileにする
    val randomAccessFile = RandomAccessFile(file.toFile(), "r")
    val channel = randomAccessFile.channel

    val request = CreateMultipartUploadRequest.builder()
        .bucket(BUCKET)
        .key(key)
        .build()

    val response = s3.createMultipartUpload(request)

    val uploadId = response.uploadId()

    val parts = mutableListOf<CompletedPart>()

    val multipart = ceil((size / MIN_MULTIPART_SIZE).toDouble()).toInt()

    for (i in 0..multipart) {
        val filePosition = MIN_MULTIPART_SIZE * i
        val contentLength = min(MIN_MULTIPART_SIZE, size - filePosition)

        if (contentLength <= 0) {
            break
        }

        val partNumber = i + 1
        val byteBuffer = ByteBuffer.wrap(ByteArray(contentLength.toInt()))
        channel.read(byteBuffer, filePosition)

        val req = UploadPartRequest.builder()
            .bucket(BUCKET)
            .key(key)
            .uploadId(uploadId)
            .contentLength(contentLength)
            .partNumber(partNumber)
            .build()

        val etag = s3.uploadPart(
            req,
            RequestBody.fromByteBuffer(byteBuffer)
        ).eTag()

        val part = CompletedPart.builder()
            .partNumber(partNumber)
            .eTag(etag)
            .build()
        parts += part
    }

    val completedMultipartUpload = CompletedMultipartUpload.builder()
        .parts(parts)
        .build()

    val completedMultipartUploadRequest = CompleteMultipartUploadRequest.builder()
        .bucket(BUCKET)
        .key(key)
        .uploadId(uploadId)
        .multipartUpload(completedMultipartUpload)
        .build()

    s3.completeMultipartUpload(completedMultipartUploadRequest)
}

companion object {
    private const val BUCKET = "foo-bucket"
    private const val URL = "http://localhost:4566"
    private const val MB: Long = 1024L * 1024L
    private const val MIN_MULTIPART_SIZE = 5L * MB
}

Bucket Lifecycle

Multipart Uploadを利用する場合 AbortIncompleteMultipartUpload rule を設定していたほうが安全 (らしい) です

参考

Work with Amazon S3 objects - AWS SDK for Java

AWS::S3::Bucket Rule - AWS CloudFormation