Affects Version/s: 6.0.12 EE, 7.0.0 M3
Environment:Tomcat 7 + Oracle 11gR2. Portal 6.0.12
Tomcat 7 + Oracle 11gR2. Portal Master GIT ID: 61058da8d9d95da84e425fb49c6b3bc6e79d3d95
Backported to Branch:Committed
Steps to reproduce
- Create a folder in DL called Test.
- Upload a single 1-byte text file (test.txt) to folder Test with 50 parallel user sessions. Each user session should upload the file 100 times and then modify 5 times. Thus, you should have 5000 files in folder Test after the whole process finishes.
- After the test has started, monitor what the DB sessions are doing during the upload with the following SQL.
Checkpoint: You should see that vast majority of wait time (>90%) will be spent on event enq: TX - row lock contention, which means that that record in table DLFolder which represents folder Test is being updated concurrently. (See sample output in the attached excel sheet.)
In methods addFileEntry() and updateFileEntry() of class DLFileEntryLocalServiceImpl the following piece of code is responsible for causing this lock contention.
As you can see, records in DLFolder are updated upon every file entry change, thus serializing concurrent requests.
Updates to DLFolder.lastPostDate will be made asynchronous.
Notes for QA
- There are basically two ways to verify that the issue has been fixed indeed.
- One is that you can see above (A), which needs some preparation and an easier one (B).
- If you choose (A), on trunk you'll see that lock contention takes only ~50% of query execution time, whilst on 6.0.x it's much worse ~95%. This means that if you see "only" ~50%, you still reproduced the issue.
Easy steps to reproduce (B)
- Making the update of lastPostDate asynchronous means that this action is handled by a background thread.
- Having this in mind you'll have to verify that.
- You don't necessarily have to use Oracle, you can use MySQL as well.
1) Create a folder Test, look up its ID in the DB.
2) Execute the following query in order to lock that row in the DB which represents folder Test.
3) Upload a document to folder Test.
Checkpoint: You should see that the document has been uploaded instantly to that folder, which means that even though the corresponding row of folder Test is locked in the DB, updating its last post date isn't happening right at that time when you upload something to it; instead it takes place in the background.
4) Take a thread dump and you'll see that one thread is hanging, because it isn't able to update that row, because you placed an explicit lock on it.