首页
学习
活动
专区
圈层
工具
发布
社区首页 >问答首页 >Cosmosdb存储过程得到的文档比实际的少

Cosmosdb存储过程得到的文档比实际的少
EN

Stack Overflow用户
提问于 2017-07-26 06:46:35
回答 2查看 431关注 0票数 1

我正在准备Javascript在cosmosdb上的存储过程,但是它获得的文档比收集的文档的实际数量要少。

C#调用sproc,C#传递一个参数"transmitterMMSI“,该参数也是该集合的分区键。

首先,在sproc中执行以下查询:

代码语言:javascript
复制
var query = 'SELECT COUNT(1) AS Num FROM AISData a WHERE a.TransmitterMMSI="' + transmitterMMSI + '"';

结果是响应输出,值为5761,与集合中文档的实际数量相同。

但是,当我将查询更改为以下内容时:

代码语言:javascript
复制
var query = 'SELECT * FROM AISData a WHERE a.TransmitterMMSI="' + transmitterMMSI + '"';

documents.length输出为5574,比实数小。

我已经更改了pageSize:-1,这意味着无限。

我做了一些谷歌搜索和堆栈溢出,似乎延续可以有所帮助。然而,我尝试了一些例子,但它们不起作用。

任何熟悉这个的人都能帮上忙吗?

下面列出脚本。

sproc脚本在这里,它也是在DownSampling.js代码中使用的文件“C#”:

代码语言:javascript
复制
function DownSampling(transmitterMMSI, interval) {
var context = getContext();
var collection = context.getCollection();
var response = context.getResponse();
var receiverTime;
var tempTime;
var groupKey;
var aggGroup = new Object();

var query = 'SELECT * FROM AISData a WHERE a.TransmitterMMSI="' + transmitterMMSI + '"';
var accept = collection.queryDocuments(collection.getSelfLink(), query, { pageSize: -1},
    function (err, documents, responseOptions) {
        if (err) throw new Error("Error" + err.message);

        // Find the smallest deviation comparting to IntervalTime in each group
        for (i = 0; i < documents.length; i++) {
            receiverTime = Date.parse(documents[i].ReceiverTime);
            tempTime = receiverTime / 1000 + interval / 2;
            documents[i].IntervalTime = (tempTime - tempTime % interval) * 1000;
            documents[i].Deviation = Math.abs(receiverTime - documents[i].IntervalTime);

            // Generate a group key for each group, combinated of TransmitterMMSI and IntervalTime
            groupKey = documents[i].IntervalTime.toString();
            if (typeof aggGroup[groupKey] === 'undefined' || aggGroup[groupKey] > documents[i].Deviation) {
                aggGroup[groupKey] = documents[i].Deviation;
            }
        }

        // Tag the downsampling
        for (i = 0; i < documents.length; i++) {
            groupKey = documents[i].IntervalTime;
            if (aggGroup[groupKey] == documents[i].Deviation) {
                documents[i].DownSamplingTag = 1;
            } else {
                documents[i].DownSamplingTag = 0;
            }

            // Remove the items that are not used
            delete documents[i].IntervalTime;
            delete documents[i].Deviation;

            // Replace the document
            var acceptDoc = collection.replaceDocument(documents[i]._self, documents[i], {},
                function (errDoc, docReplaced) {
                    if (errDoc) {
                        throw new Error("Update documents error:" + errDoc.message);
                    }
                });
            if (!acceptDoc) {
                throw "Update documents not accepted, abort ";
            } 
        } 
        response.setBody(documents.length);
    });
if (!accept) {
    throw new Error("The stored procedure timed out.");
}
} 

C#代码在这里:

代码语言:javascript
复制
    private async Task DownSampling()
    {
        Database database = this.client.CreateDatabaseQuery().Where(db => db.Id == DatabaseId).ToArray().FirstOrDefault();
        DocumentCollection collection = this.client.CreateDocumentCollectionQuery(database.SelfLink).Where(c => c.Id == AISTestCollectionId).ToArray().FirstOrDefault();

        string scriptFileName = @"..\..\StoredProcedures\DownSampling.js";
        string scriptId = Path.GetFileNameWithoutExtension(scriptFileName);

        var sproc = new StoredProcedure
        {
            Id = scriptId,
            Body = File.ReadAllText(scriptFileName)
        };

        await TryDeleteStoredProcedure(collection.SelfLink, sproc.Id);
        sproc = await this.client.CreateStoredProcedureAsync(collection.SelfLink, sproc);

        IQueryable<dynamic> query = this.client.CreateDocumentQuery(
            UriFactory.CreateDocumentCollectionUri(DatabaseId, AISTestCollectionId),
            new SqlQuerySpec()
            {
                //QueryText = "SELECT a.TransmitterMMSI FROM " + AISTestCollectionId + " a",
                QueryText = "SELECT a.TransmitterMMSI FROM " + AISTestCollectionId + " a WHERE a.TransmitterMMSI=\"219633000\"",
            }, new FeedOptions { MaxItemCount = -1, EnableCrossPartitionQuery = true, MaxDegreeOfParallelism = -1, MaxBufferedItemCount = -1 });

        List<dynamic> transmitterMMSIList = query.ToList(); //TODO: Remove duplicates
        Console.WriteLine("TransmitterMMSI count: {0}", transmitterMMSIList.Count());
        HashSet<string> exist = new HashSet<string>();

        foreach (var item in transmitterMMSIList)
        {
            //int transmitterMMSI = Int32.Parse(item.TransmitterMMSI.ToString());
            string transmitterMMSI = item.TransmitterMMSI.ToString();

            if (exist.Contains(transmitterMMSI))
            {
                continue;
            }
            exist.Add(transmitterMMSI);
            Console.WriteLine("TransmitterMMSI: {0} is being processed.", transmitterMMSI);
            var response = await this.client.ExecuteStoredProcedureAsync<string>(sproc.SelfLink,
                new RequestOptions { PartitionKey = new PartitionKey(transmitterMMSI) }, transmitterMMSI, 30);
            string s = response.Response;
            Console.WriteLine("TransmitterMMSI: {0} is processed completely.", transmitterMMSI);
        }
    }

    private async Task TryDeleteStoredProcedure(string collectionSelfLink, string sprocId)
    {
        StoredProcedure sproc = this.client.CreateStoredProcedureQuery(collectionSelfLink).Where(s => s.Id == sprocId).AsEnumerable().FirstOrDefault();
        if (sproc != null)
        {
            await client.DeleteStoredProcedureAsync(sproc.SelfLink);
        }
    }

我试图注释JS代码中的两个循环,只有documents.length输出,而响应号仍然较少。但是,我将查询更改为选择a.id是正确的。看来这是个延续的问题。

EN

回答 2

Stack Overflow用户

发布于 2017-07-26 11:43:04

sproc很可能正在安排退出时间。要在这些情况下使用延续令牌,您需要将其返回到您的C#调用代码,然后再调用传入令牌的sproc。如果你给我们看你的sproc代码,我们可以提供更多帮助。

票数 0
EN

Stack Overflow用户

发布于 2018-04-04 15:16:27

您可以使用连续令牌从sproc中重复调用queryDocuments(),而无需对客户端进行额外的往返。但是,请记住,如果这样做的次数太多,那么sproc最终会超时。在你的例子中,听起来你已经非常接近于获得你想要的所有文件,所以也许你会没事的。

下面是在sproc中使用连续令牌查询多个数据页的示例:

代码语言:javascript
复制
function getManyThings() {

    var collection = getContext().getCollection();

    var query = {
        query: 'SELECT r.id, r.FieldOne, r.FieldTwo FROM root r WHERE r.FieldThree="sought"'
    };

    var continuationToken;
    getThings(continuationToken);

    function getThings(continuationToken) {
        var requestOptions = {
            continuation: continuationToken,
            pageSize: 1000 // Adjust this to suit your needs
        };
        var isAccepted = collection.queryDocuments(collection.getSelfLink(), query, requestOptions, function (err, feed, responseOptions) {
            if (err) {
                throw err;
            }
            for (var i = 0, len = feed.length; i < len; i++) {                
                var thing = feed[i];
                // Do your logic on thing...
            }
            if (responseOptions.continuation) {
                getThings(responseOptions.continuation);
            }
            else {
                var response = getContext().getResponse();
                response.setBody("RESULTS OF YOUR LOGIC");
            }
        });
        if (!isAccepted) {
            var response = getContext().getResponse();
            response.setBody("Server rejected query - please narrow search criteria");
        }
    }
}
票数 0
EN
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接:

https://stackoverflow.com/questions/45319405

复制
相关文章

相似问题

领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档