I am trying to upload large files via a rest api to Magnolia. The project I'm working on is using Jackrabbit jcr and postgres as the db.
No issues occur for files that are less than 1GB in size. However the moment I upload larger files an error is thrown.
This is the code. The error is thrown on the Binary dataBinary = vf.createBinary(dataStream); line.
InputStream dataStream = new ByteArrayInputStream(byteArrayOutputStream.toByteArray());
ValueFactory vf = damSession.getValueFactory();
Binary dataBinary = vf.createBinary(dataStream);
This brings me to a couple of questions:
- Does the error occur because of some limit from postgres (bytea - 1GB limit?)?
- Does the error occur because of some magnolia filter (timeout?) ?
- Is there some way of increasing the max size of files in Magnolia in this specific usecase (not this one right?)/ increasing the max size in postgres?
Error stacktrace:
org.postgresql.util.PSQLException: Unable to bind parameter values for statement.
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:390) ~[postgresql-42.6.0.jar:42.6.0]
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:498) ~[postgresql-42.6.0.jar:42.6.0]
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:415) ~[postgresql-42.6.0.jar:42.6.0]
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:190) ~[postgresql-42.6.0.jar:42.6.0]
at org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:177) ~[postgresql-42.6.0.jar:42.6.0]
at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172) ~[commons-dbcp-1.4.jar:1.4]
at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172) ~[commons-dbcp-1.4.jar:1.4]
at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172) ~[commons-dbcp-1.4.jar:1.4]
at org.apache.jackrabbit.core.util.db.ConnectionHelper.execute(ConnectionHelper.java:524) ~[jackrabbit-data-2.20.9.jar:2.20.9]
at org.apache.jackrabbit.core.util.db.ConnectionHelper.reallyExec(ConnectionHelper.java:313) ~[jackrabbit-data-2.20.9.jar:2.20.9]
at org.apache.jackrabbit.core.util.db.ConnectionHelper$1.call(ConnectionHelper.java:293) ~[jackrabbit-data-2.20.9.jar:2.20.9]
at org.apache.jackrabbit.core.util.db.ConnectionHelper$1.call(ConnectionHelper.java:289) ~[jackrabbit-data-2.20.9.jar:2.20.9]
at org.apache.jackrabbit.core.util.db.ConnectionHelper$RetryManager.doTry(ConnectionHelper.java:552) ~[jackrabbit-data-2.20.9.jar:2.20.9]
at org.apache.jackrabbit.core.util.db.ConnectionHelper.exec(ConnectionHelper.java:297) ~[jackrabbit-data-2.20.9.jar:2.20.9]
at org.apache.jackrabbit.core.data.db.DbDataStore.addRecord(DbDataStore.java:360) ~[jackrabbit-data-2.20.9.jar:2.20.9]
at org.apache.jackrabbit.core.value.BLOBInDataStore.getInstance(BLOBInDataStore.java:132) ~[jackrabbit-core-2.20.9.jar:2.20.9]
at org.apache.jackrabbit.core.value.InternalValue.getBLOBFileValue(InternalValue.java:623) ~[jackrabbit-core-2.20.9.jar:2.20.9]
at org.apache.jackrabbit.core.value.InternalValue.create(InternalValue.java:379) ~[jackrabbit-core-2.20.9.jar:2.20.9]
at org.apache.jackrabbit.core.value.InternalValueFactory.create(InternalValueFactory.java:108) ~[jackrabbit-core-2.20.9.jar:2.20.9]
at org.apache.jackrabbit.core.value.ValueFactoryImpl.createBinary(ValueFactoryImpl.java:79) ~[jackrabbit-core-2.20.9.jar:2.20.9]
Magnolia version: 6.2.33
Java version: 11
EDIT 1
Completely missed this part of the error log
Caused by: java.io.IOException: Bind message length 1 073 741 889 too long. This can be caused by very large or incorrect length specifications on InputStream parameters.
at org.postgresql.core.v3.QueryExecutorImpl.sendBind(QueryExecutorImpl.java:1724) ~[postgresql-42.6.0.jar:42.6.0]
at org.postgresql.core.v3.QueryExecutorImpl.sendOneQuery(QueryExecutorImpl.java:2003) ~[postgresql-42.6.0.jar:42.6.0]
at org.postgresql.core.v3.QueryExecutorImpl.sendQuery(QueryExecutorImpl.java:1523) ~[postgresql-42.6.0.jar:42.6.0]
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:360) ~[postgresql-42.6.0.jar:42.6.0]