Posts by Flyboy

    We can send the same images between multiple Conquest servers, to KPACS and IQview without any problems.

    Just to this one server it is causing issues

    I tried even doubling the server memory to 32GB, but doesn't make a difference. Not coming anywhere close to using all the memory.


    This is the debug output:


    Looks from here the decompress might be taking longer than the timeout on the receiving end.

    While the job is running one of the CPU cores seems to be maxed out and some memory being used (some being 1-3GB) -> see screenshot attached

    So the first spike in traffic seems to be the first image which converts really quickly, followed by nothing while it is waiting on image conversion if I'm reading this correct. And the receiving end times out while waiting.

    Guessing there is no flag for multi-core compressing and decompressing?

    Also tried disabling read-ahead since I thought it might have been converting multiple things in memory, but that doesn't seem to be the case and did not help.


    I'm guessing my options now are to ask the receiving end to increase their time-outs or to invest in a faster CPU.

    Yes, the studies we are sending are 3D mammograms, so around 800MB a piece or more.

    They are being converted from j2 to ul or un.

    When I try to pass them off as j2 to the receiving party it still does a conversion to j1.

    So there is always a conversion going on, but I didn't think the decompressing of a single image took 15 seconds, causing the timeout. The server has enough horsepower to do it pretty quickly. Is there anyway to check on those timings?

    I have come across this as well.

    We are storing everything in j2 compression, which is a lot smaller than uncompressed. But some of our review stations and clients have issues with certain images, so the only good way for us to send them is uncompressed.

    In the local office, that is no issue, but for remote sites this was unworkable.


    So to fix this, we set up a Conquest server on each of the machines that will accept all images as j2, and forward them uncompressed to client.

    Works very well for us and sped up transfers a lot.

    At the blue mark on the screenshot, is where the remote server sent an abort command due to time-out.

    Before the abort command it looks to me that one of the images finishes transferring (line 14984)

    After which the next one starts (line 15985)

    Followed by the transfer and ackn of 1 packet, after which everything stops until the remote server times out and cancels the transfer.

    Looks like Conquest just stops sending for some reason, but can't find anything in its logs, not even with full debug enabled

    The d2 release is acting like the b version again.

    So our normal images are transferring.

    I'm still having an issue transferring large BTO images from Conquest to a new cloud gateway we are dealing with.

    It is failing due to a read time-out on the gateway side.


    in 1.4.19b and now 1.4.19d2 this is behaving the same way now.

    I attached an image of the traffic captured during the transfer.

    All of a sudden, there is a gap there, which causes the receiving end to time-out and cancel the transfer.


    In Dicom there is nothing in the logs even with debug set to the max level.

    Both machines are on the same virtual hosts, so there is no packet loss or firewall rules in play here.

    I even turned all the offloading options, ... off on the NIC to ensure that isn't interfering.


    Since we do not have any other issues with other vendors and other systems, I always thought it was the receiving end, but seeing the wireshark captures now is not showing any new images being sent from Conquest, so guessing it might be the sending side now.


    Anything you can see or think off?

    I wanted to come back to an issue first brought up in the 1.4.19d1 thread.

    When we are transferring images from Conquest 1.4.19b we can send things to a GE linux pacs server without any problems

    But when going to 1.4.19d1 we see the following appear in the logs and transfers are cancelled:

    Quote

    2019-06-19 15:58:57 EDT ERROR-|DicomServiceTemplate:271| Exception in creating dicom dataSpecified length (27269900) of PDU exceeds limit: 1048576

    java.io.IOException: Specified length (27269900) of PDU exceeds limit: 1048576

    Reverting back to the old version fixes the problem again.

    To try and see what might be going on, I took some captures with Wireshark and I'm noticing a difference in the traffic flow.

    While the old version is putting in small PDU segments, the new one seems to throw everything under 1 large transfer as far as I can tell.

    Screenshots attached so someone more knowledgeable can have a look and hopefully let me know what is going on here.


    Let me know if more info is needed.


    Thanks in advance

    I was able to replicate it with a specific image, which was anonymized.

    So 1 images from a dicom server runnin 1.4.19d1 on a windows server to a Linux based GE system.

    In the GE logs, the main thing seems to point to:


    2019-03-12 07:37:29 EDT INFO -|DicomServiceTemplate:102| Received association request from peer [Socket[addr=/10.x.x.xx,port=62752,localport=8118]].

    2019-03-12 07:37:29 EDT INFO -|RequestHandler:82| Association request from the peer [Socket[addr=/10.x.x.xx,port=62752,localport=8118]] accepted

    2019-03-12 07:37:29 EDT INFO -|EventManager:114| Firing the event for eventID 10003

    2019-03-12 07:37:29 EDT INFO -|CognitoTokenService:93| --------- cognitoIdTokenExpired or FirstLogin... Getting new cognitoIdToken, cognitoIdentityId, openIdToken...

    2019-03-12 07:37:30 EDT ERROR-|DicomServiceTemplate:271| Exception in creating dicom dataSpecified length (27265470) of PDU exceeds limit: 1048576

    java.io.IOException: Specified length (27265470) of PDU exceeds limit: 1048576

    ...

    2019-03-12 07:37:30 EDT ERROR-|RequestHandler:108| dicomexception occured during handling the request1003

    com.ge.med.terra.tap.dm.DMException: java.io.IOException: Specified length (27265470) of PDU exceeds limit: 1048576


    Uploading the image now and I'll send the link in a private message

    Some studies seem to fail after 20 or so images, some die right away.

    I'll see that I can find an image that triggers it every time and if it still does it after anonymizing.

    If so, I'll send it over shortly.

    We installed the 1.4.19d1 update in our datacenter

    While working with GE on some transfer issues afterwards, these are the notes I got back from them:

    So it looks like there is a PDU length issue playing here as well

    We are currently running without (known) issues on 1.4.19, so it is not urgent at this time


    Thanks for looking into this!

    We just migrated to new servers and the new conquest version.
    I'm on 1.4.19a and the green color still shows up.
    No matter the compression used for communication.
    When I grab the raw .dcm files from the server, they don't show the green, but once conquest transfers them, they do

    I currently have an instance of conquest set up to forward images to a review station and change the compression on the fly while doing so to uncompressed.
    This so all internal and vpn traffic can be done with the highest compression, but the review station that can't handle this will still get the uncompressed images presented to it.


    Since we already store the images in the main conquest system, there is no need for this routing instance to also keep them.
    So I just have the following in my dicom.ini:

    Code
    1. # Configuration of rules to modify, log or reject incoming DICOM slices
    2. ImportConverters = 1
    3. ImportConverter0 = forward to RWS3; destroy


    I'm running into some issues where that RWS3 sometimes is rejecting images or when it reboots that images get missed.
    Even when an error is thrown, the images are destroyed.
    Is there a way of to retry failing images on an importconverter?
    And if not and I need to convert this to an exportconverter, is there a way to delete the images on a successful forward so this database stays mostly empty and I don't keep any extra copies of these images?


    Thanks!

    Found the command, it is dgate --display_status:


    The difference between the 2 servers is that one of them is on 1.4.17d and one is 1.4.17beta3
    Beta 3 never switches back to the J2 compression and is transferring the images correctly
    1.4.17d is sending them as J2 and they show up garbled

    I tested different transfer by replacing the dgatesop.lst files on 1 server with different versions, but could not get it to go and transfer images in j2, even though that is what is defined in the acrnemap
    So now I think it must be due to the version of dgate
    I have no idea which version of the dgate executables I'm running, and the properties don't really help.
    What is the best idea of finding out which version you are running?
    the interface and logs don't really show me, even when restarting the server


    is there something to run on the command line to show the version?

    We are having some issues with images arriving scrambled on some of our Hologic review stations
    I have spent quote a few hours troubleshooting the issue and I think I found what is causing it, but not sure what the best way forward is.
    Example of one of the scrambled images arriving: (all headers show up ok)
    [Blocked Image: https://s14.postimg.org/ypwi29esx/scrambled_image.jpg]


    The issue is only occurring from some of our conquest servers, and so far only seems to happen when we send compressed images. Compression does not seem to matter.
    If we set the compression to un the images arrive correctly.
    All servers are running 1.4.17e


    The review station is set up as:
    rws1 10.1.1.3 4006 j2


    the working conquest servers show the following in the log:
    10/4/2016 8:19:19 AM [CONQUEST1] UPACS THREAD 42951: STARTED AT: Tue Oct 04 08:19:19 2016
    10/4/2016 8:19:19 AM [CONQUEST1] Calling Application Title : "CONQUEST1 "
    10/4/2016 8:19:19 AM [CONQUEST1] Called Application Title : "CONQUEST1 "
    10/4/2016 8:19:19 AM [CONQUEST1] Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16384
    10/4/2016 8:19:19 AM [CONQUEST1] Presentation Context 0 "1.2.840.10008.5.1.4.1.2.1.2" 1
    10/4/2016 8:19:19 AM [CONQUEST1] Presentation Context 1 "1.2.840.10008.5.1.4.1.2.2.2" 1
    10/4/2016 8:19:19 AM [CONQUEST1] C-Move Destination: "rws1 "
    10/4/2016 8:19:19 AM [CONQUEST1] Number of Images to send: 23
    10/4/2016 8:19:20 AM [CONQUEST1] Accepted compression: ui
    10/4/2016 8:19:20 AM [CONQUEST1] Sending file : E:\shimages\xxxxxxxxx\1.2.840.113663.1500.1.168875912.2.1.20160303.94628.843_0001_000001_14570163012be2.v2
    10/4/2016 8:19:20 AM [CONQUEST1] [recompress]: recompressed with mode = ui (strip=1)
    10/4/2016 8:19:20 AM [CONQUEST1] Accepted compression: ui


    The compression for the transfer is changed to ui, which I can't seem to find any info on in the manual


    a server that does not work shows:
    10/4/2016 8:14:34 AM [CONQUEST] UPACS THREAD 11104: STARTED AT: Tue Oct 04 08:14:28 2016
    10/4/2016 8:14:34 AM [CONQUEST] Calling Application Title : "CONQUEST "
    10/4/2016 8:14:34 AM [CONQUEST] Called Application Title : "CONQUEST "
    10/4/2016 8:14:34 AM [CONQUEST] Application Context : "1.2.840.10008.3.1.1.1", PDU length: 16384
    10/4/2016 8:14:34 AM [CONQUEST] Presentation Context 0 "1.2.840.10008.5.1.4.1.2.1.2" 1
    10/4/2016 8:14:34 AM [CONQUEST] Presentation Context 1 "1.2.840.10008.5.1.4.1.2.2.2" 1
    10/4/2016 8:14:34 AM [CONQUEST] Presentation Context 2 "1.2.840.10008.5.1.4.1.2.3.2" 1
    10/4/2016 8:14:34 AM [CONQUEST] C-Move Destination: "RWS1 "
    10/4/2016 8:14:34 AM [CONQUEST] Number of images to send: 28
    10/4/2016 8:14:34 AM [CONQUEST] Accepted compression: ui
    10/4/2016 8:14:34 AM [CONQUEST] Sending file : F:\pacsimages\xxxxxxxxxxx\1.2.840.113619.2.227.207919569228.7858110310083250.10005_0560_000004_14755832214a0f.dcm
    10/4/2016 8:14:34 AM [CONQUEST] [recompress]: recompressed with mode = j2 (strip=0)


    When going over all the differences between the pacs servers that work and the ones giving issues, the only difference I can find is 2 different dgatesop.lst files
    The one that works is at the top and seems to be older, the one scrambling the images, is below and is newer


    Code
    1. ## DICOM Application / sop / transfer UID list.## This list is used by the CheckedPDU_Service ( "filename" ) service# class. All incoming associations will be verified against this# file.## Revision 2: disabled GEMRStorage and GECTStorage# Revision 3: extended with new sops and with JPEG transfer syntaxes# Revision 4: added Modality Worklist query##None none RemoteAE#None none LocalAE#DICOM 1.2.840.10008.3.1.1.1 applicationVerification 1.2.840.10008.1.1 sopStoredPrintStorage 1.2.840.10008.5.1.1.27 sopHardcopyGrayscaleImageStorage 1.2.840.10008.5.1.1.29 sopHardcopyColorImageStorage 1.2.840.10008.5.1.1.30 sopCRStorage 1.2.840.10008.5.1.4.1.1.1 sopDXStorageForPresentation 1.2.840.10008.5.1.4.1.1.1.1 sopDXStorageForProcessing 1.2.840.10008.5.1.4.1.1.1.1.1 sopDMStorageForPresentation 1.2.840.10008.5.1.4.1.1.1.2 sopDMStorageForProcessing 1.2.840.10008.5.1.4.1.1.1.2.1 sopDOralStorageForPresentation 1.2.840.10008.5.1.4.1.1.1.3 sopDOralStorageForProcessing 1.2.840.10008.5.1.4.1.1.1.3.1 sopCTStorage 1.2.840.10008.5.1.4.1.1.2 sopRetiredUSMultiframeStorage 1.2.840.10008.5.1.4.1.1.3 sopUSMultiframeStorage 1.2.840.10008.5.1.4.1.1.3.1 sopMRStorage 1.2.840.10008.5.1.4.1.1.4 sopMRImageStorageEnhanced 1.2.840.10008.5.1.4.1.1.4.1 sopMRStorageSpectroscopy 1.2.840.10008.5.1.4.1.1.4.2 sopRetiredNMStorage 1.2.840.10008.5.1.4.1.1.5 sopRetiredUSStorage 1.2.840.10008.5.1.4.1.1.6 sopUSStorage 1.2.840.10008.5.1.4.1.1.6.1 sopSCStorage 1.2.840.10008.5.1.4.1.1.7 sopSCStorageSingleBitMF 1.2.840.10008.5.1.4.1.1.7.1 sopSCStorageGrayscaleByteMF 1.2.840.10008.5.1.4.1.1.7.2 sopSCStorageGrayscaleWordMF 1.2.840.10008.5.1.4.1.1.7.3 sopSCStorageTrueColorMF 1.2.840.10008.5.1.4.1.1.7.4 sopStandaloneOverlayStorage 1.2.840.10008.5.1.4.1.1.8 sopStandaloneCurveStorage 1.2.840.10008.5.1.4.1.1.9 sop#WFStorageTwelveLeadECG 1.2.840.10008.5.1.4.1.1.9.1.1 sop#WFStorageGeneralECG 1.2.840.10008.5.1.4.1.1.9.1.2 sop#WFStorageAmbulatoryECG 1.2.840.10008.5.1.4.1.1.9.1.3 sop#WFStorageHemodynamic 1.2.840.10008.5.1.4.1.1.9.2.1 sop#WFStorageCardiacElectrophysiology 1.2.840.10008.5.1.4.1.1.9.3.1 sop#WFStorageBasicVoiceAudio 1.2.840.10008.5.1.4.1.1.9.4.1 sopStandaloneModalityLUTStorage 1.2.840.10008.5.1.4.1.1.10 sopStandaloneVOILUTStorage 1.2.840.10008.5.1.4.1.1.11 sopGrayscaleSoftcopyPresentationStateStorage 1.2.840.10008.5.1.4.1.1.11.1 sopRetiredXASinglePlaneStorage 1.2.840.10008.5.1.4.1.1.12 sopXASinglePlaneStorage 1.2.840.10008.5.1.4.1.1.12.1 sopRFStorage 1.2.840.10008.5.1.4.1.1.12.2 sopXABiPlaneStorage 1.2.840.10008.5.1.4.1.1.12.3 sopNMStorage 1.2.840.10008.5.1.4.1.1.20 sopVLImageStorage 1.2.840.10008.5.1.4.1.1.77.1 sopVLMultiFrameImageStorage 1.2.840.10008.5.1.4.1.1.77.2 sopVLMicroscopicSlideStorage 1.2.840.10008.5.1.4.1.1.77.3 sopVLPhotographicStorage 1.2.840.10008.5.1.4.1.1.77.4 sopVLEndoscopicImageStorage 1.2.840.10008.5.1.4.1.1.77.1.1 sopVLMicroscopicImageStorage 1.2.840.10008.5.1.4.1.1.77.1.2 sopVLSlideCoordinatesMicroscopicImageStorage 1.2.840.10008.5.1.4.1.1.77.1.3 sopVLPhotographicImageStorage 1.2.840.10008.5.1.4.1.1.77.1.4 sopBasicTextSR 1.2.840.10008.5.1.4.1.1.88.11 sopEnhancedSR 1.2.840.10008.5.1.4.1.1.88.22 sopComprehensiveSR 1.2.840.10008.5.1.4.1.1.88.33 sopMammographyCADSR 1.2.840.10008.5.1.4.1.1.88.50 sopKeyObjectSelectionDocument 1.2.840.10008.5.1.4.1.1.88.59 sopPETStorage 1.2.840.10008.5.1.4.1.1.128 sopStandalonePETCurveStorage 1.2.840.10008.5.1.4.1.1.129 sopRTImageStorage 1.2.840.10008.5.1.4.1.1.481.1 sopRTDoseStorage 1.2.840.10008.5.1.4.1.1.481.2 sopRTStructureStorage 1.2.840.10008.5.1.4.1.1.481.3 sopRTTreatmentRecordStorage 1.2.840.10008.5.1.4.1.1.481.4 sopRTPlanStorage 1.2.840.10008.5.1.4.1.1.481.5 sopRTBrachyTreatmentRecordStorage 1.2.840.10008.5.1.4.1.1.481.6 sopRTTreatmentSummaryRecordStorage 1.2.840.10008.5.1.4.1.1.481.7 sop#GEMRStorage 1.2.840.113619.4.2 sop#GECTStorage 1.2.840.113619.4.3 sopGE3DModelObjectStorage 1.2.840.113619.4.26 sopGERTPlanStorage 1.2.840.113619.5.249 sopGERTPlanStorage2 1.2.840.113619.4.5.249 sopGESaturnTDSObjectStorage 1.2.840.113619.5.253 sopPhilips3DVolumeStorage 1.2.46.670589.5.0.1 sopPhilips3DObjectStorage 1.2.46.670589.5.0.2 sopPhilipsSurfaceStorage 1.2.46.670589.5.0.3 sopPhilipsCompositeObjectStorage 1.2.46.670589.5.0.4 sopPhilipsMRCardioProfileStorage 1.2.46.670589.5.0.7 sopPhilipsMRCardioImageStorage 1.2.46.670589.5.0.8 sopPatientRootQuery 1.2.840.10008.5.1.4.1.2.1.1 sopPatientRootRetrieve 1.2.840.10008.5.1.4.1.2.1.2 sopStudyRootQuery 1.2.840.10008.5.1.4.1.2.2.1 sopStudyRootRetrieve 1.2.840.10008.5.1.4.1.2.2.2 sopPatientStudyOnlyQuery 1.2.840.10008.5.1.4.1.2.3.1 sopPatientStudyOnlyRetrieve 1.2.840.10008.5.1.4.1.2.3.2 sopFindModalityWorkList 1.2.840.10008.5.1.4.31 sopPatientRootRetrieveNKI 1.2.826.0.1.3680043.2.135.1066.5.1.4.1.2.1.2 sopStudyRootRetrieveNKI 1.2.826.0.1.3680043.2.135.1066.5.1.4.1.2.2.2 sopPatientStudyOnlyRetrieveNKI 1.2.826.0.1.3680043.2.135.1066.5.1.4.1.2.3.2 sopBasicGrayscalePrintManagementMeta 1.2.840.10008.5.1.1.9 sopBasicColorPrintManagementMeta 1.2.840.10008.5.1.1.18 sopBasicFilmSession 1.2.840.10008.5.1.1.1 sopBasicFilmBox 1.2.840.10008.5.1.1.2 sopBasicGrayscaleImageBox 1.2.840.10008.5.1.1.4 sopBasicColorImageBox 1.2.840.10008.5.1.1.4.1 sopBasicPrinter 1.2.840.10008.5.1.1.16 sopLittleEndianImplicit 1.2.840.10008.1.2 transferLittleEndianExplicit 1.2.840.10008.1.2.1 transfer#BigEndianExplicit 1.2.840.10008.1.2.2 transferJPEGBaseLine1 1.2.840.10008.1.2.4.50 transfer LittleEndianExplicitJPEGExtended2and4 1.2.840.10008.1.2.4.51 transfer LittleEndianExplicit#JPEGExtended3and5 1.2.840.10008.1.2.4.52 transfer LittleEndianExplicitJPEGSpectralNH6and8 1.2.840.10008.1.2.4.53 transfer LittleEndianExplicit#JPEGSpectralNH7and9 1.2.840.10008.1.2.4.54 transfer LittleEndianExplicitJPEGFulllNH10and12 1.2.840.10008.1.2.4.55 transfer LittleEndianExplicit#JPEGFulllNH11and13 1.2.840.10008.1.2.4.56 transfer LittleEndianExplicitJPEGLosslessNH14 1.2.840.10008.1.2.4.57 transfer LittleEndianExplicit#JPEGLosslessNH15 1.2.840.10008.1.2.4.58 transfer LittleEndianExplicit#JPEGExtended16and18 1.2.840.10008.1.2.4.59 transfer LittleEndianExplicit#JPEGExtended17and19 1.2.840.10008.1.2.4.60 transfer LittleEndianExplicit#JPEGSpectral20and22 1.2.840.10008.1.2.4.61 transfer LittleEndianExplicit#JPEGSpectral21and23 1.2.840.10008.1.2.4.62 transfer LittleEndianExplicit#JPEGFull24and26 1.2.840.10008.1.2.4.63 transfer LittleEndianExplicit#JPEGFull25and27 1.2.840.10008.1.2.4.64 transfer LittleEndianExplicit#JPEGLossless28 1.2.840.10008.1.2.4.65 transfer LittleEndianExplicit#JPEGLossless29 1.2.840.10008.1.2.4.66 transfer LittleEndianExplicitJPEGLossless 1.2.840.10008.1.2.4.70 transfer LittleEndianExplicit#JPEGLS_Lossless 1.2.840.10008.1.2.4.80 transfer LittleEndianExplicit#JPEGLS_Lossy 1.2.840.10008.1.2.4.81 transfer LittleEndianExplicit#RLELossless 1.2.840.10008.1.2.5 transfer LittleEndianExplicit#LittleEndianExplicitDeflated 1.2.840.10008.1.2.1.99 transfer LittleEndianExplicitJPEG2000LosslessOnly 1.2.840.10008.1.2.4.90 transfer LittleEndianExplicitJPEG2000 1.2.840.10008.1.2.4.91 transfer LittleEndianExplicit



    I'm not really sure what in here might be breaking the transfers, but if someone could help out that would be great.
    Or point me to a possible other cause for this issue.


    And we would need the later version of the file to support BTO images I think, so reverting all servers to the older file might not be an option.


    David