QSqlQuery::size() not returning actual number?

[davecotter]

i have execute this select statement:

"SELECT * FROM sqlite_master WHERE type = 'table' AND name = '[tablename]'"

then i ask how many rows i got back (i want to know if this table exists)
but size() returns -1.
here's my workaround, which works, but WHY?

int	Result::CountRows()
{
	int	resultI(0);

	if (IsOk()) {

		if (i_qquery.isSelect()) {
		
			if (i_qquery.isActive()) {
				int	sizeI = i_qquery.size();
				
				if (sizeI >= 0) {
					resultI = sizeI;
					
				} else {
					i_qquery.last();
			
					sizeI = i_qquery.at();
					
					if (sizeI >= 0) {
						resultI = sizeI + 1;
					}
				}

			} else {
				CF_ASSERT(0);
			}
		} else {
			resultI = i_qquery.numRowsAffected();
		}
	}

	return resultI;
}

[davecotter]

see @JonB 's answer above! thanks!

[SGaist]

Hi,

I don't remember if it's still the case for SQLite but IIRC not all database engines support returning the query size. You should check that before going further.

[JonB]

@davecotter
As @SGaist has just posted, size() returns -1 if the driver does not support returning the count of the result set.

Assuming that's the case,

then i ask how many rows i got back (i want to know if this table exists)

"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = '[tablename]'"

will just return one row containing the number you want (I would assume either 0 or 1 from your query), you might prefer to use that?

[SGaist]

You can use QSqlDriver::hasFeature to check for that.

[davecotter]

@JonB I’m not sure I understand your reply, I already am using that select statement. My q is: what is the proper way to ask how many rows were returned?

[Kent-Dorfman]

As Gaist mentioned, getting the number of rows from a query response is a very haphazard process when you try to implement it for any ole generic database. MANY DBC backends cache returned rows in manageable groups and use something called a cursor to traverse through them. It is possible, and probable, that the real number of rows returned is far more than is in the currently cached response.

I recommend you used the count() SQL aggregate function to find out how many rows would be returned from your query. It is always guaranteed to return a 1 row result. Looks like JonB also recommended this.

Worst case is that you have to do two queries. First one to find out the size of your result set.

[davecotter]

@Kent-Dorfman sorry if I wasn’t clear at the start, this is only SQLite and will never change

[Kent-Dorfman]

@davecotter said in QSqlQuery::size() not returning actual number?:

@Kent-Dorfman sorry if I wasn’t clear at the start, this is only SQLite and will never change

doesn't matter...using the count() method is the more universal way of doing it.

[davecotter]

@Kent-Dorfman I’m not at all versed at this: is there example code showing how to use count()?

[Kent-Dorfman]

yes, plenty, but I have no resources off the top of my head. look at any Qt SQL examples you can find. The key thing is that your quere should be:

select count(*) as cnt from sqlite_master WHERE type = 'table' AND name = '[tablename]'

and grab the cnt field from the response

[davecotter]

@Kent-Dorfman I’m trying to understand “grab the field from the response” ...

[Kent-Dorfman]

a single field row is returned (called cnt). look at the examples and find one where you get the value of a returned field. there are a few ways to do this.

Executing your QSqlQuery will return a QResultSet object. The field can be accessed using boundValue(0) method of QResultSet. Ya gotta do some googling and research if you want any more detail than that.

[mrjj]

hi
One way could be. ( note my table is called person. change to yours)

QSqlQuery query("select count(*) as cnt from sqlite_master where type='table' and name='person'");
if (query.next())
    qDebug() << query.value(0).toInt();

However, in org post you said (i want to know if this table exists)
so just as a note
using SQLite version 3.3+
you can say
"create table if not exists TableName ..:"
if you want to check if table exits and create it if not.
Might not be your use case, but its good to know.

[davecotter]

sorry, i guess i should have made my question more generic than "how many rows will be returned from this specific statement". what i should have asked is:

if i've already performed some statement, and i only have access to the QSqlQuery at this point and no longer know what the actual statement was, how do i ask how many rows that returns? I want it to work for any statement, not just one that asks if a table exists. is my solution just grossly inefficient? is there a better way than this:

  			i_qquery.last();
  	
  			sizeI = i_qquery.at();
  			
  			if (sizeI >= 0) {
  				resultI = sizeI + 1;
  			}

[Christian Ehrlicher]

As @SGaist already mentioned this only works (and unrelated from Qt) when the underlying database supports this. You can check it with QSqlDriver::hasFeature ...

[davecotter]

yes we have determined that "size()" doesn't work on SQLite, i understand that.

my question has evolved into: is the way i'm doing it now a bad / terribly inefficient way to do it?
for reference, this is what i'm now doing:

i_qquery.last();
		
sizeI = i_qquery.at();
				
if (sizeI >= 0) {
	resultI = sizeI + 1;
}

thanks

[Christian Ehrlicher]

It's not very efficient but when you really need the result size for whatever reason you've no other chance.

[Kent-Dorfman]

@davecotter said in QSqlQuery::size() not returning actual number?:

yes we have determined that "size()" doesn't work on SQLite, i understand that.

my question has evolved into: is the way i'm doing it now a bad / terribly inefficient way to do it?

That's already been answered as well...

[JonB]

@davecotter said in QSqlQuery::size() not returning actual number?:

@JonB I’m not sure I understand your reply, I already am using that select statement. My q is: what is the proper way to ask how many rows were returned?

No, you must look at what I wrote: it's SELECT COUNT(*) instead of SELECT *. And you retrieve that single result number as the first column in the one & only row returned as the result set, and you must know how to read stuff out of the result set else you couldn't be using queries for much.

Let's just summarise something else:

if i've already performed some statement, and i only have access to the QSqlQuery at this point and no longer know what the actual statement was, how do i ask how many rows that returns? I want it to work for any statement, not just one that asks if a table exists.

Yes, if you have previously executed an arbitrary query and now wish to know how many rows it returned you have no choice but to use your method.

is my solution just grossly inefficient? is there a better way than this:

Well, yes, it is inefficient (though unavoidable if you want the apples for other purposes too). Think of this: the SQL server has a bunch of apples. You ask the server to send you all the apples, and you sit & count how many have arrived to get the answer. (It's not the counting per se which is slow, it's the having to send & receive all the apples in order to count them which is bad.) It's not perfect, is it? Let's hope it has not sent you a lot of apples to count!

A much more efficient way is to ask the server to do the counting at its end, not send all the apples, but send a single number saying how many there are. That is what SELECT COUNT(...) does. It's great for getting a quick total of the apples, but quite useless if you want to look at each one for some other reason.

[davecotter]

see @JonB 's answer above! thanks!

[kshegunov]

@JonB said in QSqlQuery::size() not returning actual number?:

It's great for getting a quick total of the apples

Yes, with some exceptions where a "quick total" isn't exactly quick ... ;)

[JonB]

@kshegunov
That's unfair! :) Give me any situation where COUNT() at the server is not vastly quicker than passing row sets to client to count? Give me any situation where COUNT() at server is not vastly quicker than code at server to iteratively count rows? Care to comment on whether SELECT COUNT(*) FROM table, with no WHERE condition like OP's example, allows provider to use huge optimization like it has already maintained total table row count and just returns that? Look, there's "quick" and there's "quick", this one is "quick" but I did not say "instantaneous" or "like lightning" :)

[kshegunov]

@JonB said in QSqlQuery::size() not returning actual number?:

Give me any situation where COUNT() at the server is not vastly quicker than passing row sets to client to count?

That'd be all the situations where the server is running a InnoDB engine, for example, and the table is much larger than the resultset.

Give me any situation where COUNT() at server is not vastly quicker than code at server to iteratively count rows?

No such situation exists, that's why aggregates exist. Unless you have a stored procedure that does something else, that is, then the counting can be a "byproduct".

Care to comment on whether SELECT COUNT(*) FROM table, with no WHERE condition like OP's example, allows provider to use huge optimization like it has already maintained total table row count and just returns that?

See above. InnoDB doesn't have the real row count at hand. It can give you an approximate amount of rows based on the allocated pages quickly, but for a real row count time of execution is significant, compared to some other engines like (my)ISAM. Anyway, it's more complicated than to claim speed directly.

Look, there's "quick" and there's "quick"

Yes, we agree on that one.

[JonB]

@kshegunov said in QSqlQuery::size() not returning actual number?:

Give me any situation where COUNT() at the server is not vastly quicker than passing row sets to client to count?

That'd be all the situations where the server is running a InnoDB engine, for example, and the table is much larger than the resultset.

I admit I know nothing about InnoDB [I use MySQL, so I guess I use that?], so your points are fair enough. (For the avoidance of doubt, I am talking about where there is a database "server" involved.) But I don't get this one at all? How can it possibly be quicker to pass whatever all the results are (with or without a condition) to the client to count than to count them at the server side? If you're going to go:

SELECT * FROM table WHERE smaller-resultset-condition

and then send to count them at the client, why is that quicker than

SELECT COUNT(*) FROM table WHERE smaller-resultset-condition

?

[kshegunov]

@JonB said in QSqlQuery::size() not returning actual number?:

If you're going to go

Because this is not what you usually do when dealing with mid-size to large datasets.
Say for the sake of argument you count the number of rows to provide pagination of some sort, which is a typical case. You count, init the pages' count, current page number or w/e and then retrieve only a single small result set from the table. Right, so you count all relevant records just for the simple paging.
Now, this is okay most of the time, at the end of the day that's why you have a database server (or engine, or w/e) - to provide storage. It gets a bit more involved when you have concurrent transactions over complex queries, however; i.e. you have snapshots all over the place.
Say you count the rows of a query that has a nested query and altogether them queries span 8-10 tables over some JOINs. The count can get real expensive in these cases, especially if you consider why you do it to begin with - to fetch some meta information about the dataset and just provide some small part of it afterwards as actual data. That's what I referred to in my tease.