Project

General

Profile

Actions

Incident #258

closed

TabMan - the files associated to the resources are downloaded without extension

Added by Gianpaolo Coro almost 10 years ago. Updated about 9 years ago.

Status:
Closed
Priority:
High
Assignee:
Fabio Sinibaldi
Category:
-
Target version:
Start date:
Jun 15, 2015
Due date:
% Done:

100%

Estimated time:
Infrastructure:
Production

Description

After producing an image via StatMan, the resources attached to the table have correct names but they are downloaded as files without extension.
Please, note that a suggested file name is provided by StatMan, along with a Java Image Object that can be dumped with a mime-type associated (decided by the dumper).
Thus, you could either use the name suggested by StatMan and guess the mime-type from the name, or you could dump it with another name, after deciding for a mime-type to associate.

Actions #1

Updated by Pasquale Pagano almost 10 years ago

  • Assignee changed from Fabio Sinibaldi to Giancarlo Panichi
  • Priority changed from Normal to High
Actions #2

Updated by Giancarlo Panichi almost 10 years ago

  • Status changed from New to In Progress

If we determine that any resource that reaches the portlet has a name without an extension and a mimetype correct, then I can add the extension to the file name. I do some tests to see if it is always so.

Actions #3

Updated by Fabio Sinibaldi almost 10 years ago

We cannot assume here that file names always have or always miss extension (I think also it depends on the particular algorithm in case of SM integration), so the portlet should check it before eventually adding the extension to the filename. I think it's a better approach to adjust the names in the operation that handles it before presenting it to the portlet (or where it is generated, where possible).
Please note also that in case of SM integration, mimetype is only available for resources retrieved from the workspace (SMType IMAGES).

Actions #4

Updated by Giancarlo Panichi almost 10 years ago

  • Assignee changed from Giancarlo Panichi to Gianpaolo Coro
  • % Done changed from 0 to 100

One possible solution has been realized and is put on dev portal.
GP can check if it is okay.

Actions #5

Updated by Gianpaolo Coro almost 10 years ago

  • % Done changed from 100 to 50

The files are still downloaded without extension. Probably the mime-type is still not associated to the images produced by the statistical manager.

Actions #6

Updated by Gianpaolo Coro almost 10 years ago

  • Assignee changed from Gianpaolo Coro to Fabio Sinibaldi
Actions #7

Updated by Giancarlo Panichi almost 10 years ago

The problem is that a invalid MimeType is set. PNG image has MimeType "image/png", instead is set "png".

ResourceDescriptorTD [id=152, name=Chart focused on decimallatitude - Distribution of summed quantities over emerged lands, divided per country, 
description=Charts, 
creationDate=2015-07-01 11:17, 
creatorId=10001, 
resourceType=Generic File, 
resourceTD=InternalURITD [id=5593cc52e4b00f093ea5d524, mimeType=png, thumbnailTD=null]]
Actions #8

Updated by Fabio Sinibaldi almost 10 years ago

Gianpaolo, can you provide a list of expected mimetypes that we could check/fix when we import resources onto Tabular Data?
If not we could :

  • proceed by evidence
  • use an external library that analyses the imported file and gives us the proper mimetype (definitely more secure but not optimal because we need to move all files from the storage to the Tabular data service)
Actions #9

Updated by Gianpaolo Coro almost 10 years ago

The files currently produced by StatMan are:
png
csv
gif
zip
txt
no-extension (binary)

The images produced by the algorithms are always generic Java "Image" objects. They are always saved by StatMan as PNGs on the workspace. Thus, there will be no JPG or other formats for images.

The modification asked for this ticket regards the PNGs only, as they should be visualized as images and downloaded as PNGs. These are reported as IMAGE content by StatMan (thus they are recognizable also from the category) and are the only kind of image reported by the service. Anyway, the mime-type should be correctly reported by the workspace, isn't it?

The other files are classified as FILEs by StatMan and their mime-type can be retrieved from their extension. These files always have an extension, unless they are binary files. In this case mime-type is not really important, because these are expected to be treated as generic files.

Actions #10

Updated by Massimiliano Assante almost 10 years ago

  • Status changed from In Progress to Feedback
  • Assignee changed from Fabio Sinibaldi to Gianpaolo Coro
  • % Done changed from 50 to 90

Gianpaolo can you please verify that this incident was fixed in production after the rollout of 3.8.0?

Actions #11

Updated by Gianpaolo Coro almost 10 years ago

The images now work, but I see problems in other kinds of files:
String results are reported with a generic STRING name and not with their real associated key, whereas the Files object names are reported with their associated keys and not with the file name produced by the statistical manager.

E.g. the output of "Feed Forward Ann" is a "General Results.txt" file plus a binary file. The TXT should contain the string "Learning threshold=0.56" and instead contains " [STRING] : 0.56". Furthermore, the binary file accompanying this TXT should have the name introduced by the user as input, instead, as result on TabMan, it is transformed into a generic "Trained Neural Network" string. This is the key of the File object sent by the Statistical Manager but it is not the file name (which has instead the name introduced by the user).

Actions #12

Updated by Gianpaolo Coro over 9 years ago

  • Assignee changed from Gianpaolo Coro to Giancarlo Panichi
Actions #13

Updated by Giancarlo Panichi over 9 years ago

  • Assignee changed from Giancarlo Panichi to Fabio Sinibaldi
Actions #14

Updated by Fabio Sinibaldi over 9 years ago

  • % Done changed from 90 to 100

The problem has been fixed and will be released in 3.10 release.
I've also added the experiment title as a suffix for the general result file in order to distinct between the various generated ones.

Actions #16

Updated by Fabio Sinibaldi about 9 years ago

  • Status changed from Feedback to Closed

No feedback received, I suppose I can close this one.

Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 8.91 MB)