Issue 127464 - The file '<file name>' could not be repaired and therefore can not be opened
Summary: The file '<file name>' could not be repaired and therefore can not be opened
Status: CLOSED NOT_AN_OOO_ISSUE
Alias: None
Product: Calc
Classification: Application
Component: open-import (show other issues)
Version: 4.1.3
Hardware: PC Linux 64-bit
: P5 (lowest) Critical (vote)
Target Milestone: ---
Assignee: AOO issues mailing list
QA Contact:
URL:
Keywords: needhelp
Depends on:
Blocks:
 
Reported: 2017-07-03 03:01 UTC by jari.kosonen@rayence.com
Modified: 2017-07-10 12:20 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
Latest Confirmation in: ---
Developer Difficulty: ---


Attachments
can not save the document (5.58 KB, image/gif)
2017-07-03 03:01 UTC, jari.kosonen@rayence.com
no flags Details
can not open the document (4.02 KB, image/gif)
2017-07-03 03:01 UTC, jari.kosonen@rayence.com
no flags Details
Sample how the images are used (with blank images) (105.57 KB, application/pdf)
2017-07-06 23:27 UTC, jari.kosonen@rayence.com
no flags Details

Note You need to log in before you can comment on or make changes to this issue.
Description jari.kosonen@rayence.com 2017-07-03 03:01:26 UTC
Created attachment 86167 [details]
can not save the document
Comment 1 jari.kosonen@rayence.com 2017-07-03 03:01:43 UTC
Created attachment 86168 [details]
can not open the document
Comment 2 jari.kosonen@rayence.com 2017-07-03 03:04:49 UTC
I think the Sheet2 of this document that has some images can not save properly
and is possibly accumulating the images from previous save, like if it can not
clear the sheet data completely. Even the Sheet2 is deleted from the ods-file
it still has very large size, like the images still would be in the ods-file.

[jari@vatcad3 IMG]$ ls -la *.ods
-rw-r--r-- 1 jari TEAM1 3566162876 Jul  3 10:29 SORT_IMAGES_SUDS150_WN174958_Jul2017.ods
Comment 3 jari.kosonen@rayence.com 2017-07-03 06:55:32 UTC
The file is possibly too large to be uploaded there... I might try to upload it now.
Comment 4 Peter 2017-07-03 07:00:57 UTC
You Cam Upload the Document to File Share Services like Dropbox, google Docs ans provide a Link.

All the best
peter
Comment 5 Peter 2017-07-03 07:09:35 UTC
Also, also please keep in mind that we have here not the focus to fix your issue. If we can confirm it is a general issue and if we can locate it we will improve the codebase. Which will then be released.

Sometimes it is better to check with user Mailing List or user Forums. To see if there is a workaround.

We would be still happy receive some information on the issue but more from the overall improvement perspective.

Hope you understand.

All the best 
Peter
Comment 7 jari.kosonen@rayence.com 2017-07-03 07:27:17 UTC
I try manage, even it does not work...
The size of >3Gb is little bit suspicious.
I think it is adding more data into the sheet everytime I used it.
Even I delete the Sheet2 and it looks clean, the data is probably
still in the ods-file somewhere.
Comment 8 Peter 2017-07-03 07:31:31 UTC
I agree 3 GB is suspicious.

Please describe from where you load data?

I am traveling. Can have a look tonight.
Comment 9 jari.kosonen@rayence.com 2017-07-03 08:23:44 UTC
Can you receive the file? I am not 100% sure if the sharing works.
Comment 10 Peter 2017-07-03 08:36:02 UTC
It did not work.
Comment 11 jari.kosonen@rayence.com 2017-07-04 02:30:21 UTC
https://drive.google.com/open?id=0BwII5b2Vli_RUW9tcVFkOEdvSEU
Comment 12 jari.kosonen@rayence.com 2017-07-04 02:36:14 UTC
Please try the latest google-drive document download.
Due to large size it may have problems.
Comment 13 jari.kosonen@rayence.com 2017-07-06 01:11:21 UTC
Can you download the file from the google drive?
Comment 14 Keith N. McKenna 2017-07-06 03:28:55 UTC
I was able to download the ods file from the google drive link and can confirm that AOO 4.1.3 reports it as corrupted and tries and fails to repair it. The contents.xml file shows a lot of what appear to be graphics frame entries, but it will take someone with more knowledge than I decipher the root cause.
Comment 15 Keith N. McKenna 2017-07-06 03:30:09 UTC
(In reply to Keith N. McKenna from comment #14)
> I was able to download the ods file from the google drive link and can
> confirm that AOO 4.1.3 reports it as corrupted and tries and fails to repair
> it. The contents.xml file shows a lot of what appear to be graphics frame
> entries, but it will take someone with more knowledge than I decipher the
> root cause.

I can confirm this with Windows 10 and AOO 4.1.3
Comment 16 Regina Henschel 2017-07-06 13:15:08 UTC
The problem is, that the pictures are used for filling. After use, the fill-style definition like
<draw:fill-image draw:name="WN14B736_5f_2_5f_X30Y30_5f_2" draw:display-name="WN14B736_2_X30Y30_2" xlink:href="Pictures/1000000000000654000008B6D97E5E90.bmp" xlink:type="simple" xlink:show="embed" xlink:actuate="onLoad"/>
stays in the subdocument styles.xml.

On load AOO tries to open all those files, which are currently count 625. Each of them is 3.613.678 Bytes, approx 3,44MB. That results in 2150MB approx 2,1GB. At least for my PC that's far too much and I get a "Bad allocation" error because of out of RAM.

The message "cannot be repaired" is misleading.

Whatever you want to do, that's not the way to go.

You likely need to modify your macros. I don't know, what your macros will do. But the pictures look like you do not examine them manually. Do you have considered not to include the pictures into the sheet, but only enter the results into the sheet?
Comment 17 jari.kosonen@rayence.com 2017-07-06 23:27:46 UTC
Created attachment 86171 [details]
Sample how the images are used (with blank images)
Comment 18 jari.kosonen@rayence.com 2017-07-06 23:34:23 UTC
I don't know what means:
"The problem is, that the pictures are used for filling. After use, the fill-style definition like <draw:fill-image draw:name="WN14B736_5f_2_5f_X30Y30_5f_2" draw:display-name="WN14B736_2_X30Y30_2" xlink:href="Pictures/1000000000000654000008B6D97E5E90.bmp" xlink:type="simple" xlink:show="embed" xlink:actuate="onLoad"/>
stays in the subdocument styles.xml."
Could you inform to me how to remove it from there?
Even if I deleted the 2nd sheet the file size did not reduce, that is the visible sheet seemed empty, but almost all the data remained in the file.

This document is to show the pictures. So it is not possible to enter the results only in this case. The results are then in another sheet already. So that works, but I can not get the image summary.
Pictures are inserted there with macro as:

  oLinked=Doc.createInstance("com.sun.star.drawing.GraphicObjectShape")
  url$=sPath+"/"+sFile 
  surl$=sDie+"_"+Trim(Str(fi-43))
  If oBitmaps.hasByName(surl) Then
    oBitmaps.removeByName(surl)
  End If
  aPos.X=(Sheet.getCellByPosition((Nx+2)*i+pX+1,(Ny+3)*j+pY+6).Position.X)+rX
  aPos.Y=(Sheet.getCellByPosition((Nx+2)*i+pX+1,(Ny+3)*j+pY+6).Position.Y)+rY

  oBitmaps.insertByName(surl,url)
  oLinked.GraphicURL=converttourl(url$)
  rem oLinked.Anchor=Sheet.getCellByPosition((Nx+2)*i+pX+1,(Ny+3)*j+pY+6)

  c=oDrawPage.count
    
  oLinked.RotateAngle=rot
  oLinked.ShearAngle=0
  If mirrorX="Y" Then
    oLinked.IsMirrored=True
  Else
    oLinked.IsMirrored=False
  End If
  oLinked.Position=aPos
  oLinked.Size=aSize
  oLinked.Name=sDie
  oDrawPage.add(oLinked)
Comment 19 jari.kosonen@rayence.com 2017-07-06 23:36:04 UTC
It is possible that the macro has some errors, since I did not find maybe good reference to make such macro.
Comment 20 jari.kosonen@rayence.com 2017-07-07 05:42:17 UTC
How to generate the contents.xml -file?
Could you upload it here?
Comment 21 Peter 2017-07-07 05:56:16 UTC
Regina says you have to much pictures per document. We currently can not hold that much pictures.
 How much memory do you have in your machine?
Comment 22 jari.kosonen@rayence.com 2017-07-07 05:58:48 UTC
The machine has the maximum memory, that is 64Gb for the Intel(R) Xeon(R) CPU E5-1660 v2 @ 3.70GHz:
[jari@vatcad3 IMG]$ free
             total       used       free     shared    buffers     cached
Mem:      65900004   14911552   50988452      11280     168496   11130816
-/+ buffers/cache:    3612240   62287764
Swap:     33021948       9828   33012120
Comment 23 Regina Henschel 2017-07-07 13:27:56 UTC
The parts with oBitmaps look suspicious. An object ("com.sun.star.drawing.BitmapTable") is used for background filling. But it looks as if you only want to add an image to the draw page of the sheet, not to use this image as background. Try to comment out statements, which use oBitmaps. You might need to adapt pathes, from where the images are loaded by the macro.

As a general rule for huge images, only work with linked images and do not manipulate the images in OpenOffice but use a dedicated application for that. For example, do not rotate an image in OpenOffice, but rotate it outside and then insert the rotated image into OpenOffice as link.

[Work on a copy of the file!] If you need to remove the images from an existing file, then you can unpack the file. It is actually a zip-container. Then open the sub-document styles.xml in an editor and remove all the <draw:fill-image ... lines.
Open the subfolder Pictures and remove all pictures. After that mark all the content and pack it.
You might want to use the scripts (Linux or Cygwin) posted in https://lists.oasis-open.org/archives/office/201706/msg00001.html to unpack and pack it.

Using linked images needs to load at least those images, which are currently visible. Have you already adapt the settings in Tools > Options > OpenOffice, item Memory, to your huge amount of images and to the huge size of the images?

Do you need all of these images at the same time? Perhaps you can divide the problem in smaller tasks?

For to learn Basic use the book from Andrew Pitonyak.
Comment 24 jari.kosonen@rayence.com 2017-07-08 01:39:59 UTC
I'll try to check how to insert the image by the macro as a link instead of embedded image. But it seems hard to be found.
Comment 25 jari.kosonen@rayence.com 2017-07-10 03:14:19 UTC
Maybe at the moment I could not find the right book to add the images as a link. I have just the "OpenOffice.org" macros explained, and that book is not too good to explain how to insert the bitmaps into excel sheet.
Comment 26 jari.kosonen@rayence.com 2017-07-10 04:05:17 UTC
It seems that the images are not embedded into Sheet2, but into the document and deleting Sheet2 does not remove the images from the document:
https://drive.google.com/open?id=0BwII5b2Vli_RMDQ5ekZISllzeUk
This is maybe because the images are bitmaps (that you said are the background images) that are attached to the document itself instead of Sheet2.
Comment 27 Peter 2017-07-10 04:49:02 UTC
We get OT here. For macro coding help please ask on Forum or on Mailing List. There are more people that can help.
This is only the development team looking here, and we are quite small Team.
Comment 28 jari.kosonen@rayence.com 2017-07-10 04:54:57 UTC
I think the problems is that it accumulate the same file again and again with different internal name. It can not check if the image is there already. But maybe it should be checked by the macro instead of the calc.
Comment 29 jari.kosonen@rayence.com 2017-07-10 04:59:32 UTC
It works now, after removed the oBitmap definitions. I think that was done to insert the images as embedded. But I think the method is wrong way to try insert embedded image. I try further check about the issue on the Forum. Thanks for the support given.
  oLinked.Position=aPos
  oLinked.Size=aSize
  oLinked.Name=sDie
  oLinked.AsLink=False
  
  oDrawPage.add(oLinked)

The ".AsLink" does not work in this context. I try further check with the Forum.
Comment 30 Peter 2017-07-10 05:59:17 UTC
You are welcome. I see this as resolved.