host
ho
!

Examples:

sql /nolog

-- Full command name
SQL> host ls
dist        README.md    src

-- Short name
SQL> ho ls
dist        README.md    src

-- Really short name: (optional space between ! and the command you want to run)
SQL> !ls
dist        README.md    src

Personally I like using the shortcut ! reference as it’s very quick to write when doing a lot of scripting.

The host command can be really useful is when working with the new SQLcl CI/CD tool Projects. You can quickly see the status of the git repo using !git status rather than having to exit SQLcl, run the command, then go back into SQLcl.

If an application item is constant for the entire application (regardless of user/session) then it may not make sense for a new row to be created to store a session state value. For example, suppose you had the following setup:

• Application item called APP_NAME

• Set for each session based on a value in a lookup table

• In dev you set to Demo Dev, test: Demo Test, and production: Demo

This means that the following would occur for every session:

• Computation or process on new instance to load value from lookup table

• A new row is created in apex_application_items for every session

The above steps would happen for every session despite the fact that the value does not change. In smaller systems this may not matter however on higher volume systems or environments with lots of applications this may add additional overhead.

Solution

To avoid the additional overhead you can use a Text Message. Text Messages are primarily intended for translation purposes however they also allow developers to have configurable application constants. Text messages are different than Application Settings in the sense that they can be referenced in APEX as a substitution string.

The following example shows how to use text messages for this purpose.

Go to Shared Components > Text Messages

Click Create Text Message and add the following:

On the login page (or this can be anywhere in the application where you want a dynamic application name) change the region title to: &APP_TEXT$APP_NAME. More information about referencing text messages as substitution strings can be found in docs.

When the page is run it shows Demo Dev as expected:

To change this in other environments it's unrealistic to set the value via the APEX application builder as most developers won't have access to Test and Prod instances. Instead it can be easily set through the API apex_lang.update_message As part of your release process after the APEX application has been update the following code should be run to set the value:

declare
    c_apex_app_id apex_application_translations.application_id%type := 200;
    l_translation_entry_id apex_application_translations.translation_entry_id%type;
begin

    select 
        aat.translation_entry_id
    into 
        l_translation_entry_id
    from apex_application_translations aat
    where 1=1
        and aat.application_id = c_apex_app_id
        and aat.translatable_message = 'APP_NAME'
    ;

    -- Detect if in APEX already
    -- Usually necessary as this should be run as a post-install script in SQLcl
    -- 
    -- If you're not in an APEX application you'll get following error:
    -- ORA-20001: Package variable g_security_group_id must be set.
    -- 
    -- May get ORA-20987: APEX - An API call has been prohibited.
    -- If so go to Shared Components >  Security Attributes
    -- Check box: Runtime API Usage > Modify This Application
    -- 
    if apex_application.g_flow_id is null then 
        apex_session.create_session (
            p_app_id => c_apex_app_id,
            p_page_id => 1,
            p_username => 'dummy'
        );
    end if;

    -- Set text message:
    apex_lang.update_message (
        p_id => l_translation_entry_id,
        -- Note: This value should come from a lookup table
        p_message_text => 'Dev Test'
    );

    -- Commit is required for changes to take affect
    commit;
end;
/

After running this script the login page will show the update application name:

Closing Remarks

I've used this technique in a very high-use application that can have between 300,000 ~ 500,000 sessions a day. The application has about 10 application constants. If Application Items were used for these constants it would result in 10 x 500,000 (5 Million) unnecessary extra writes occurring every day to store values that don't change.

I also found that there were some places "regular" substitution strings did not work (using an Application Item or Page Item) but worked for Text Messages. These were extreme edge cases.

When this technique is used you need to ensure that you have a good release process. I.e. right after the APEX application is installed/updated in each environments the text messages need to be updated as they will contain the "original" text message from the application export.

I've had the fortunate opportunity of beta testing the SQL Dev extension for VSC for a while and as such have a few helpful hints to start using it. Before continuing this article please read Jeff Smith's VSC announcement article as it covers a lot of the key features.

Keyboard Shortcuts

Traditional SQL Developer users are used to the standard keyboard shortcuts (ex F11 for commit). You can view the SQL Developer for VSC shortcuts by going to Code > Settings > Keyboard Shortcuts

The new tab will show all the shortcuts available in VSC. Filter down to oracle sql developer You'll see the list of default shortcuts that you can modify. There's also a lot of commands that don't have shortcuts associated to them which you can configure as needed.

If you can't remember all the keyboard shortcuts you can easily search on the fly using VSC Command Palette and type in the command you're looking for.

Extension Settings

It's always good to look at all the options for each VSC extension that is installed. To view the SQL Dev VSC extension settings go to Extensions then search for @installed sql > click on the gears and select Extension Settings:

Some of key settings to pay attention to:

• Database › Nls: Sort I like using BINARY_CI more information about this can be found here.

• Connections › Startup Script: Can reference your login.sql More about login.sql here.

• Database > Date / Timestamp Formats: I like to include minutes and seconds as well as displaying the month as a string (use Mon ) as shown below

Substitution Strings

If you're running an anonymous PL/SQL block or a query with an ampersand (& ) using the run command (or ctrl+enter) it will prompt you for a the substitution string:

To get around this just use Compile command (open the command palette and search for it). This will run just the block / query that the cursor is in (not the entire sheet).

Query Results: Column Width

If your query has a lot of columns you'll need to select a formatting option. Right click on the row header and select Auto-fit All Columns and select the best option.

Feedback

Given that this is the initial release of SQL Dev for VSC there's a lot of opportunity for community engagement You can post your feedback, suggestions, bugs, etc on the SQL Developer for VS Code Forums.

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
   <url>
      <loc>http://www.example.com/</loc>
      <lastmod>2005-01-01</lastmod>
      <changefreq>monthly</changefreq>
      <priority>0.8</priority>
   </url>
</urlset>

I've seen various implementations of this in SQL and PL/SQL that require a lot of code. Thankfully the entire XML file can be generated with one SQL statement. Using the sample emp table the following query shows how it can be generated: note the URLs won't make sense given the emp table doesn't map to any URLs:

select
    xmlserialize(
        content
        xmlroot(
            xmlelement(
                "urlset",
                xmlattributes('http://www.sitemaps.org/schemas/sitemap/0.9' as "xmlns"),
                xmlagg(
                    xmlelement(
                        "url",
                        xmlelement("loc", 'http://my-url.com/' || e.ename),
                        -- See https://www.w3.org/TR/NOTE-datetime for format
                        -- If e.hiredate was a timestamp can use:
                        -- xmlelement("lastmod", to_char(e.hiredate at time zone 'UTC', 'YYYY-MM-DD"T"HH24:MI:SSTZH:TZM')),
                        xmlelement("lastmod", to_char(e.hiredate, 'YYYY-MM-DD')),
                        -- See https://www.sitemaps.org/protocol.html#xmlTagDefinitions for changefreq and other options
                        xmlelement("changefreq", 'daily')
                    )
                ) -- xmlagg
            ) -- urlset
        ,
        -- Verion is only meant for a number but using a trick to also add the "encoding" attribute
        -- See https://forums.oracle.com/ords/apexds/post/how-to-add-version-and-encoding-to-xml-2163#comment_323462166473746547632690547881602100073
        version '1.0" encoding="UTF-8'
        )
        as clob
    ) sitemap_xml
from emp e
;

Result:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>http://my-url.com/KING</loc>
    <lastmod>1981-11-17</lastmod>
    <changefreq>daily</changefreq>
  </url>
<!-- ... (all the other emps) -->
  <url>
    <loc>http://my-url.com/MILLER</loc>
    <lastmod>1982-01-23</lastmod>
    <changefreq>daily</changefreq>
  </url>
</urlset>

For APEX developers see APEX idea FR-3105 has some discussion on how to include a sitemap.xml file in an application. Specifically look at the comment by vladislav.uvarov

One misconception is that all references to this page item will be null / won't do anything. This is not correct as page items that aren't marked to be rendered can still have values. In the example below I have two page items:

• P6_ALWAYS has no conditions and will always be shown on the page

• P6_NEVER its condition will be set to Never and not render on the page

• Both of the page items have a item source as a Function Body with the following code:

-- For always for P6_ALWAYS is "always" and P6_NEVER is "never"
logger.log('Source for always/never');
return 'always/never';

When the page is run it looks like:

Things to note:

• P6_NEVER was not rendered

• No values in session state (i.e. P6_ALWAYS item's value will only be saved in session state when the page is saved)

• Looking at the logs only the P6_ALWAYS source code section was run

  

A new computation is now added to the page to set a value for P6_NEVER as shown below:

When the page is run it will look the same as before except that P6_NEVER has a value in session state. This means that when referencing P6_NEVER in a block of SQL or PL/SQL it will have a value, however if trying to reference it on the page (via JavaScript) it does not exists nor has a value.

In the past when I've explained this to people (both new and experienced APEX developers) and they think it's a bug. It isn't for the following reasons:

• APEX has an order of operations. Essentially it runs "top down" for everything listed on the left side in Page Designer

• Since Pre-Rendering is done before page items are "processed" APEX can't know if a page item should not be rendered

  • This can also be true of a page item's condition references another page ex. For example if a page item's condition is P6_ADMIN_YN = 'Y' then APEX will only know if this is true or false once it starts to process the page item.

In a previous article I covered how to do case insensitive sorting in APEX. Before continuing please read the article as this post builds upon it. This post shows how to do case insensitive searching by default in APEX (i.e. every time you have a where clause MARTIN is the same as martin).

Suppose an report's query is the following. No row's will be returned because all the jobs are uppercase. In this example the valid job name is CLERK not cLERk.

select  *
from emp
-- Note this could also be a page item like :P1_JOB
-- Where user types in job name
where job = 'cLERk'

To have all queries do case insensitive searching in APEX go to Shared Components > Globalization Attributes and change the following settings:

• Character Value Comparison: BINARY_CI

• Character Value Comparison Behavior: Linguistic

If you run the same report with the above query it will now return rows for all the CLERK jobs:

Reference Material

To learn more about each of the key APEX settings please click on the help icon as it covers what each option means. The settings reference NLS parameters NLS_SORT and NLS_COMP. You can find the default settings for each parameter using the query below. More information about these parameters can be found in the Oracle documentation for NLS parameters.

select *
from v$nls_parameters
where parameter in ('NLS_COMP', 'NLS_SORT')
;

PARAMETER VALUE  CON_ID 
--------- ------ ------ 
NLS_SORT  BINARY      0 
NLS_COMP  BINARY      0

If you don't want to enable case insensitive searching for your entire application but do want it on a specific query/report you can do so using collate binary_ci. Chris Saxon has a great article which goes into detail about the collate command.

select  *
from emp
where job = 'cLERk' collate binary_ci

The Problem

I recently had a data integrity issue with an Oracle materialized view (MV) in where no errors were raised but the data was not correct in the MV. The following situation is what triggered caused the issue:

• MV with fast refresh

• Underlying query contains an aggregate function (min, max, sum, count, etc)

• An update or delete occurs on base table

The following code snippet highlights the issue:

-- reset
-- drop table demo_emps;
-- drop materialized view demo_emps_mv;

create table demo_emps (
    id number primary key, 
    name varchar2(255),
    sal number
);

insert into demo_emps values(1, 'martin', 100);

-- Required for fast refresh
create materialized view log on demo_emps
with rowid, sequence(id, sal)
including new values
;

-- Will return the max sal
create materialized view demo_emps_mv
refresh fast on commit
as
select max(sal) max_sal
from demo_emps
;

-- Returns 100 (correct)
select *
from demo_emps_mv
;

MAX_SAL 
------- 
    100 

-- Insert a new row
insert into demo_emps values(2, 'sally', 200);

-- Commit required for MV Refresh:
commit;

-- Returns 200 (correct)
select *
from demo_emps_mv
;

MAX_SAL 
------- 
    200 

-- View MV Status
select staleness
from user_mviews
where 1=1
    and mview_name = 'DEMO_EMPS_MV'
;

STALENESS 
--------- 
FRESH     

-- Update rows (add 1 to each salary)
-- Same issue will also occur if a row is deleted
update demo_emps
set sal = sal + 1
;

commit;

-- Should show 201 but shows old 200 (incorrect)
select *
from demo_emps_mv
;

MAX_SAL 
------- 
    200 

-- View MV Status
-- Note: Staleness now shows that it is UNUSABLE
select staleness
from user_mviews
where 1=1
    and mview_name = 'DEMO_EMPS_MV'
;

STALENESS 
--------- 
UNUSABLE

Despite no errors raised the data is clearly not correct. Some thoughts:

• I think the only way to resolve this issue is to create the MV as refresh on demand and do manual refreshes

• It is can be very hard to catch this bug and I recommend monitoring your MVs to ensure the data is correct

  • Update: You can catch this issue but need to analyze the MV (see below)

This issue is documented in MOS 726333.1 Why Does Mview Staleness in Dba_Mview Become Unusable?

For a MV with MIN/MAX or COUNT we only support fast refresh if DML is only insert. On delete/update we won't be able to do fast refresh. For such on-commit mview refresh case we mark it as UNUSABLE.

The Solution

This solution is an excerpt from Connor McDonald sent me.

This can be resolved by modifying the MV to also include count(*) (specifically count(*) and not anything else). Using the above examples:

-- Drop the MV objects 
drop materialized view log on demo_emps;
drop materialized view demo_emps_mv;

-- Rebuilding MV log to include name (not part of fix but needed for grouping)
create materialized view log on demo_emps
with rowid, sequence(id, sal, name)
including new values
;

create materialized view demo_emps_mv
refresh fast on commit
as
select
    name,
    max(sal) max_sal,
    -- Key to solving the issue
    count(*) cnt
from demo_emps
-- Needs to have a "group by" clause (most MVs will)
group by name
;

select *
from demo_emps_mv
;

NAME   MAX_SAL CNT 
------ ------- --- 
martin     101   1 
sally      201   1 

-- Before this update triggered the issue
update demo_emps
set sal = sal + 5
;

commit;

-- Now values in MV are correct
select *
from demo_emps_mv
;

NAME   MAX_SAL CNT 
------ ------- --- 
martin     106   1 
sally      206   1

Explanation

Going back to the first block of code if I rebuild the MV and run dbms_mview.explain_mview to analyze the MV. Some key notes:

• 5006: MV supports insert

• 6007 MV does not other DML

• 6008 Reason why other DML is not supported / solution

drop materialized view demo_emps_mv;

create materialized view demo_emps_mv
refresh fast on commit
as
select max(sal) max_sal
from demo_emps
;

-- Used to store results of dbms_view.explain_mview
-- See https://docs.oracle.com/en/database/oracle/oracle-database/19/arpls/DBMS_MVIEW.html#GUID-DB06F5D8-C883-4443-9123-E1519B44AFA6
truncate table mv_capabilities_table;

exec dbms_mview.explain_mview('DEMO_EMPS_MV');

select 
    seq,
    capability_name,
    possible,
    related_text,
    msgtxt
from mv_capabilities_table
order by seq
;


  SEQ CAPABILITY_NAME               POSSIBLE RELATED_TEXT MSGTXT                                                                             
----- ----------------------------- -------- ------------ ---------------------------------------------------------------------------------- 
    1 PCT                           N                                                                                                        
 1002 REFRESH_COMPLETE              Y                                                                                                        
 2003 REFRESH_FAST                  Y                                                                                                        
 3004 REWRITE                       N                                                                                                        
 4005 PCT_TABLE                     N        DEMO_EMPS    relation is not a partitioned table                                                
 -- INSERTS ARE OK
 5006 REFRESH_FAST_AFTER_INSERT     Y                                                                                                        
 -- OTHER DML NOT OK
 6007 REFRESH_FAST_AFTER_ONETAB_DML N                     mv uses the MIN, MAX or ANY_VALUE aggregate functions                              
 6008 REFRESH_FAST_AFTER_ONETAB_DML N                     COUNT(*) is not present in the select list                                         
 7009 REFRESH_FAST_AFTER_ANY_DML    N                     see the reason why REFRESH_FAST_AFTER_ONETAB_DML is disabled                       
 8010 REFRESH_FAST_PCT              N                     PCT is not possible on any of the detail tables in the materialized view           
 9011 REWRITE_FULL_TEXT_MATCH       N                     query rewrite is disabled on the materialized view                                 
10012 REWRITE_PARTIAL_TEXT_MATCH    N                     query rewrite is disabled on the materialized view                                 
11013 REWRITE_GENERAL               N                     query rewrite is disabled on the materialized view                                 
12014 REWRITE_PCT                   N                     general rewrite is not possible or PCT is not possible on any of the detail tables 
13015 PCT_TABLE_REWRITE             N        DEMO_EMPS    relation is not a partitioned table

Updating the MV to use the solution (with count(*)) and running the same analysis has the following results:

• Lines 6007 and 6008 aren't present anymore

• REFRESH_FAST_AFTER_ANY_DML is now marked as Y

drop materialized view demo_emps_mv;

create materialized view demo_emps_mv
refresh fast on commit
as
select
    name,
    max(sal) max_sal,
    count(*) cnt
from demo_emps
group by name
;

truncate table mv_capabilities_table;

exec dbms_mview.explain_mview('DEMO_EMPS_MV');

select 
    seq,
    capability_name,
    possible,
    related_text,
    msgtxt
from mv_capabilities_table
order by seq
;


  SEQ CAPABILITY_NAME            POSSIBLE RELATED_TEXT MSGTXT                                                                             
----- -------------------------- -------- ------------ ---------------------------------------------------------------------------------- 
    1 PCT                        N                                                                                                        
 1002 REFRESH_COMPLETE           Y                                                                                                        
 2003 REFRESH_FAST               Y                                                                                                        
 3004 REWRITE                    N                                                                                                        
 4005 PCT_TABLE                  N        DEMO_EMPS    relation is not a partitioned table                                                
 5006 REFRESH_FAST_AFTER_INSERT  Y                                                                                                        
 7007 REFRESH_FAST_AFTER_ANY_DML Y                                                                                                        
 8008 REFRESH_FAST_PCT           N                     PCT is not possible on any of the detail tables in the materialized view           
 9009 REWRITE_FULL_TEXT_MATCH    N                     query rewrite is disabled on the materialized view                                 
10010 REWRITE_PARTIAL_TEXT_MATCH N                     query rewrite is disabled on the materialized view                                 
11011 REWRITE_GENERAL            N                     query rewrite is disabled on the materialized view                                 
12012 REWRITE_PCT                N                     general rewrite is not possible or PCT is not possible on any of the detail tables 
13013 PCT_TABLE_REWRITE          N        DEMO_EMPS    relation is not a partitioned table

select job, max(sal) max_sal
from emp
group by job
order by job
;

JOB       MAX_SAL 
--------- ------- 
ANALYST      3000 
CLERK        1300 
MANAGER      2975 
PRESIDENT    5000 
SALESMAN     1600

The next logical question is: Who earns the max salary? A very common solution to this problem is to use an analytical function:

select
    job, ename, sal
from (
    select
        row_number() over (partition by job order by sal desc) rn,
        job, ename, sal
    from emp
)
where rn = 1
order by job
;

JOB       ENAME   SAL 
--------- ------ ---- 
ANALYST   SCOTT  3000 
CLERK     MILLER 1300 
MANAGER   JONES  2975 
PRESIDENT KING   5000 
SALESMAN  ALLEN  1600 

-- Explain plan:                                                                                     
------------------------------------------------------------------------------------ 
| Id  | Operation                   | Name | Rows  | Bytes | Cost (%CPU)| Time     | 
------------------------------------------------------------------------------------ 
|   0 | SELECT STATEMENT            |      |    14 |   546 |     4  (25)| 00:00:01 | 
|*  1 |  VIEW                       |      |    14 |   546 |     4  (25)| 00:00:01 | 
|*  2 |   WINDOW SORT PUSHED RANK   |      |    14 |   252 |     4  (25)| 00:00:01 | 
|   3 |    TABLE ACCESS STORAGE FULL| EMP  |    14 |   252 |     3   (0)| 00:00:01 | 
------------------------------------------------------------------------------------ 

Predicate Information (identified by operation id):                                  
---------------------------------------------------                                  

   1 - filter("RN"=1)                                                                
   2 - filter(ROW_NUMBER() OVER ( PARTITION BY "JOB" ORDER BY "SAL")<=1)

There is an alternate way to do this using the keep clause and dense_rank along with the first/last commands:

select
    job,
    -- The initial "min" aggregate could also be "max" 
    -- The key logic is in the dense_rank first/last and ordering
    min(ename) keep (dense_rank last order by sal) ename,
    -- Could also write as:
    -- min(ename) keep (dense_rank first order by sal desc) ename,
    max(sal)
from emp
group by job
order by job
;

JOB       ENAME  MAX(SAL) 
--------- ------ -------- 
ANALYST   FORD       3000 
CLERK     MILLER     1300 
MANAGER   JONES      2975 
PRESIDENT KING       5000 
SALESMAN  ALLEN      1600 

-- Explain Plan
----------------------------------------------------------------------------------- 
| Id  | Operation                  | Name | Rows  | Bytes | Cost (%CPU)| Time     | 
----------------------------------------------------------------------------------- 
|   0 | SELECT STATEMENT           |      |     5 |    90 |     4  (25)| 00:00:01 | 
|   1 |  SORT GROUP BY             |      |     5 |    90 |     4  (25)| 00:00:01 | 
|   2 |   TABLE ACCESS STORAGE FULL| EMP  |    14 |   252 |     3   (0)| 00:00:01 | 
-----------------------------------------------------------------------------------

Using the dense_rank command is easier to write and it may result in more performant queries. The different explain plans shows that less bytes were required.

Note: the keen observer may note that for the ANALYST position there's two different results. That's because both they're two analysts (FORD and SCOTT) and they both earn the same amount. If "ties are allowed" the following query can be used:

select job, ename, sal
from (
    select 
        job, ename, sal,
        dense_rank() over (partition by job order by sal desc) rn
    from emp
)
where rn = 1
order by job
;

-- Note that ANALYST has two rows given that SCOTT and FORD earn the same amount
JOB       ENAME   SAL 
--------- ------ ---- 
ANALYST   SCOTT  3000 
ANALYST   FORD   3000 
CLERK     MILLER 1300 
MANAGER   JONES  2975 
PRESIDENT KING   5000 
SALESMAN  ALLEN  1600

References

• Oracle's official documentation on this feature is actually in the first command

• Rob van Wijk has a good article about the keep clause

  • Covers some of the performance gains in detail

Thanks to Jeff Kemp for showing me this feature!

create table demo_json (
    id number,
    json_data varchar2(4000)
);

insert into demo_json(id, json_data)
values(1, '{
    "name": "Martin",
    "company": "Oracle",
    "colors": ["red", "white", "blue"],
    "age": 25,
    "index": "demo_index"
}');
commit;

select
    json_value(d.json_data, '$.name') name,
    json_value(d.json_data, '$.colors[2]') third_color,
    json_value(d.json_data, '$.age.number()') age
from demo_json d
;

NAME   THIRD_COLOR AGE 
------ ----------- --- 
Martin blue         25

Oracle has the ability to use simple dot-notation access to json data. They're two requirements for this to work:

• The column type must either be type json or it must have an is json check constraint.

• Must alias the table when selecting (in the demo using d as table alias name)

-- Highlight that a check constraint is 
select d.json_data.name
from demo_json d
;


00904. 00000 -  "%s: invalid identifier"
*Cause:    
*Action:
Error at Line: 1 Column: 6
;

-- Add check constraint
alter table demo_json add constraint demo_json_ck1 check (json_data is json);

select 
    d.json_data.name,
    d.json_data.colors[2] third_color,
    d.json_data.age.number() age
from demo_json d
;

NAME   THIRD_COLOR AGE 
------ ----------- --- 
Martin blue         25

It's recommended to read the documentation as it covers all the features about JSON dot notation in SQL.

Thanks to Stefan Dobre for pointing this feature out to me.

If you spend a lot of time with code (SQL and PL/SQL) or settings in Page Designer you may tend to spend more of your time on the right hand side of the page. For example report queries, PL/SQL code blocks, settings, etc. In this case you may want move the sections around so that the settings are in the main pane of Page Designer. The video below highlights how to do this. I also move help text to the right pane so each time I click on a field I can quickly see inline help text.

To reset the layout click on the Utilities (wrench) > Layout > Reset Layout as shown below.

apex_string.split

apex_string.split allows you to split a delimited string into rows (similar procedures exists for numbers and clobs).

select s.column_value
from apex_string.split('emp,dept', ',') s
;

COLUMN_VALUE 
------------ 
emp          
dept

Pipelined Functions

Pipelined functions are PL/SQL functions that "pipe rows" and are selectable as a table in a SQL. apex_string.split is a pipelined function. Oracle Base has a great article all about pipelined table functions which you're encouraged to read and learn more about.

Pickler Fetch

You may have seen COLLECTION ITERATOR PICKLER FETCH in an Oracle explain plan but never understood what they are. Martin Widlake has a great post about this here. TL;DR Oracle is processing something of unknown number of rows and needs to provide the optimizer with a number of rows. It defaults to 8168 rows.

apex_string.split, Pipelined Functions, and Pickler Fetches

Putting it altogether now. When using apex_string.split in your queries you could be introducing a performance problem without even realize it. The following example highlights this.

-- Create a large table
-- This may take a while to run
create table all_objs as 
select object_name
from all_objects
;

-- If using SQL Dev Web run the next three all at the same time
alter session set optimizer_capture_sql_plan_baselines=true;

select ao.object_name
from all_objs ao
    join apex_string.split('EMP,DEPT',',') s on 1=1
        and s.column_value = ao.object_name
;

select plan_table_output
from table(dbms_xplan.display_cursor(null,null,'typical'))
;

SQL_ID  cq8vm0pb4jacn, child number 0
-------------------------------------
select ao.object_name from all_objs ao     join 
apex_string.split('EMP,DEPT',',') s on 1=1         and s.column_value = 
ao.object_name

Plan hash value: 1741181693

------------------------------------------------------------------------------------------------
| Id  | Operation                           | Name     | Rows  | Bytes | Cost (%CPU)| Time     |
------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                    |          |       |       |   112 (100)|          |
|*  1 |  HASH JOIN                          |          | 13550 |   529K|   112   (1)| 00:00:01 |
|   2 |   JOIN FILTER CREATE                | :BF0000  |  8168 | 16336 |    29   (0)| 00:00:01 |
|   3 |    COLLECTION ITERATOR PICKLER FETCH| SPLIT    |  8168 | 16336 |    29   (0)| 00:00:01 |
|   4 |   JOIN FILTER USE                   | :BF0000  | 46481 |  1724K|    82   (0)| 00:00:01 |
|*  5 |    TABLE ACCESS STORAGE FULL        | ALL_OBJS | 46481 |  1724K|    82   (0)| 00:00:01 |
------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - access("AO"."OBJECT_NAME"=VALUE(KOKBF$))
   5 - storage(SYS_OP_BLOOM_FILTER(:BF0000,"AO"."OBJECT_NAME"))
       filter(SYS_OP_BLOOM_FILTER(:BF0000,"AO"."OBJECT_NAME"))

The plan from the example above highlights that the overall cardinality estimate is 13550 rows.

If you know the approximate number of items in the apex_string.split then a SQL hint can be applied to the query to tell the optimizer what to expect. The following demo highlights this. Using the previous example, it is a known fixed list of two items.

select 
    -- Note "s" refers to the table alias below
    /*+ cardinality (s, 2) */
    ao.object_name
from all_objs ao
    join apex_string.split('EMP,DEPT',',') s on 1=1
        and s.column_value = ao.object_name
;

select plan_table_output
from table(dbms_xplan.display_cursor(null,null,'typical'));


SQL_ID  80h36ywn1rjjn, child number 0
-------------------------------------
select      -- Note "s" refers to the table alias below     
/*+ cardinality (s, 2) */     ao.object_name from all_objs ao     join 
apex_string.split('EMP,DEPT',',') s on 1=1         and s.column_value = 
ao.object_name

Plan hash value: 1741181693

------------------------------------------------------------------------------------------------
| Id  | Operation                           | Name     | Rows  | Bytes | Cost (%CPU)| Time     |
------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                    |          |       |       |   111 (100)|          |
|*  1 |  HASH JOIN                          |          |     3 |   120 |   111   (0)| 00:00:01 |
|   2 |   JOIN FILTER CREATE                | :BF0000  |     2 |     4 |    29   (0)| 00:00:01 |
|   3 |    COLLECTION ITERATOR PICKLER FETCH| SPLIT    |     2 |     4 |    29   (0)| 00:00:01 |
|   4 |   JOIN FILTER USE                   | :BF0000  | 46481 |  1724K|    82   (0)| 00:00:01 |
|*  5 |    TABLE ACCESS STORAGE FULL        | ALL_OBJS | 46481 |  1724K|    82   (0)| 00:00:01 |
------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - access("AO"."OBJECT_NAME"=VALUE(KOKBF$))
   5 - storage(SYS_OP_BLOOM_FILTER(:BF0000,"AO"."OBJECT_NAME"))
       filter(SYS_OP_BLOOM_FILTER(:BF0000,"AO"."OBJECT_NAME"))


-- Cleanup
drop table all_objs;

The key difference in this that the optimizer now expects 3 rows (in the HASH JOIN row) instead of 13550! When doing more complex joins this can really improve performance.

In an APEX form where the user may be selecting a several objects I tend to use orders of magnitude estimate (ex: 10, 100, 1000) for a hint.

A few key notes about SQL hints:

• Just because you can doesn't mean you should. Use then when necessary (and in the above example it is justified as the optimizer has no way of knowing the expected number of rows)

• Good intro article about SQL hints by David Njoku here

Oracle regular expressions (regexp) support both [[:upper:]] or [A-Z] to find uppercase letters. At first glance they appear the same. Even regex101.com defines them as the same:

[[:upper:]]: Matches uppercase letters. Equivalent to [A-Z]. The double square brackets is not a typo, POSIX notation demands it.

There is a slight difference between the two. [A-Z] only deals with the 26 letters in the English alphabet whereas [[:upper:]] deals with special alphabet characters such as Ê - accent circumflex (or as we learned in French glass "e avec un chapeau "). The following example highlights the differences using the demo Oracle emp table:

-- Change the "A" in Martin to A with an accent on it
update emp
set ename = 'MÄRTIN'
where empno = 7654;


-- [A-Z]
select *
from emp
where 1=1
    and empno = 7654
    and regexp_like(ename, '^[A-Z]+$')
;

-- Returns
/*
No data found
*/

-- [[:upper:]]
select ename
from emp
where 1=1
    and empno = 7654
    and regexp_like(ename, '^[[:upper:]]+$')
;

-- Returns:
/*
ENAME  
------ 
MÄRTIN 
*/


-- Look at ASCII characters
select ename, dump(ename)
from emp
where empno = 7654
;

/*
ENAME  DUMP(ENAME)                         
------ ----------------------------------- 
MÄRTIN Typ=1 Len=7: 77,195,132,82,84,73,78 
*/

-- You can see the second characters out of normal a-Z ASCII characters

-- Reset
update emp
set ename = 'MARTIN'
where empno = 7654
;

As you can see the results are different and [[:upper:]] matched the special characters. The following description from this Stackoverflow post highlights the differences:

[A-Z] matches only an ASCII uppercase letter, that is, a letter from A through Z. There are other, non-ASCII uppercase letters (e.g., in languages other than English).

If you use regular expressions in your code do not go change everything from [A-Z] without consideration. They're some times where you may want to keep it in place (example lookup codes, etc). I tend to use [[:upper:]] for when dealing with user inputed fields when it makes sense.

with 
    data as (
        -- Generates:
        -- - Numbers each 20 digits in length 
        -- - Alpha numeric strings 20 characters in length
        select 
            trunc(dbms_random.value() * power(10, 20)) num,
            dbms_random.string('x', 20) str
        from dual
        connect by level <= 10000
    )
select 
    listagg(d.num, ',') within group (order by d.num asc) all_nums,
    listagg(d.str, ',') within group (order by d.num asc) all_strs
from data d
;

-- Raises
01489. 00000 - "result of string concatenation is too long"
*Cause: String concatenation result is more than the maximum size.
*Action: Make sure that the result is less than the maximum size.
Error at Line: 0 Column: 0

You can control whether an error is raised (on overflow error) or truncate (on overflow truncate ...) the string using the listagg_overflow_clause. The truncate solution still doesn't solve the issue if you have a lot of content you want to concatenate.

An alternative approach is to use json_arrayagg along with the returning clob option. This trick only works in the following conditions:

• Numbers, dates, etc (i.e. structured date types)

• For strings they have some known format and do not contain quotes or commas

The example above can be converted from listagg to json_arrayagg:

with 
    data as (
        select 
            trunc(dbms_random.value() * power(10, 20)) num,
            dbms_random.string('x', 20) str
        from dual
        connect by level <= 10000
    )
select 
    json_arrayagg(d.num returning clob) json_array_nums,
    json_arrayagg(d.str returning clob) json_array_strs
from data d
;

-- Returns 
-- [16390942703930704741,72925476174569629179,98810344353460307813,...]
-- ["OJV24DVTF6EIF80F76NE","CY6T5L8S3BZW2CN0CHJY",...]

The result still contains the square brackets and quotes around the strings. To remove them to just have comma delimited list we can use regular expressions (can also do with a few replace statements):

with 
    data as (
        select 
            trunc(dbms_random.value() * power(10, 20)) num,
            dbms_random.string('x', 20) str
        from dual
        connect by level <= 10000
    )
select 
    -- Note: the order of the square brackets in the regexp string: []["] matters
    -- as SQL doesn't require escaping strings within square bracket operators
    regexp_replace(json_arrayagg(d.num returning clob), '[]["]', '') nums,
    regexp_replace(json_arrayagg(d.str returning clob), '[]["]', '') strs
from data d
;

-- Returns 
-- 16390942703930704741,72925476174569629179,98810344353460307813,... 
-- OJV24DVTF6EIF80F76NE,CY6T5L8S3BZW2CN0CHJY,...

You can easily do this in SQL using the json_mergepatch function. The following example highlights how to modify and also add items to a JSON object:

select 
    json_mergepatch(
-- default value
'
{
    "foo": 123,
    "bar": {
        "firstName": "default",
        "lastName": "default"
    }
}
',
-- custom value
-- replace "lastName"
-- adds "middleName"
-- adds "language"
'
{
    "bar": {
        "lastName": "dsouza",
        "middleName": "giffy"
    },
    "language": "english"
}
'
-- If you your resulting JSON will be larger than 4000 then return a clob by uncommented below
-- returning clob
--
-- The "pretty" keyword is optional and just helps with display
pretty
) demo
from sys.dual;

The resulting JSON object is:

{
    "foo": 123,
    "bar": {
        "firstName": "default",
        "lastName": "dsouza",
        "middleName": "giffy"
    },
    "language": "english"
}

By default APEX will warn users when leaving a page. When users modify a page with input and try to close the tab or refresh the page they will be shown the following dialog (note: each browser will show a slightly different warning message):

The default options for this can be changed at the page level by toggling the Warn on Unsaved Changes option:

Alternatively you can control each page item to determine if it'll trigger a warn on unsaved message:

There are some times when you may want to manually determine if a page item has been modified. In JavaScript the apex.item object contains a function called isChanged() Using the new(ish) apex.items object (which contains all the page items) you can quickly identify which page items have been changed. The following function will return an array of APEX item names that have been changed:

Object.keys(apex.items).filter((key) => apex.items[key].isChanged());

Hayden Hudson has a neat snippet that shows some custom warnings when a page item has been changed.

-- Note the two digit year
select to_date('24-Jan-21', 'DD-MON-YYYY')
from dual;

I recently asked this same question on Twitter and the results were quite interesting given that this should be a very straight forward answer. Over 50% of the people got it wrong:

{% asset_img twitter-results.png %}

The correct answer is that the query returns: 24-Jan-0021. If you thought an error would be raised you're in good company. I reached out to several Oracle ACEs (i.e. world leading Oracle experts) and they all got it wrong as well (myself included).

By using a date format of DD-MON-YYYY if a user does not explicitly enter a four digit year, Oracle will left pad the number with 0s. I.e. 21 becomes 0021. This makes sense that one would expect 5-Jan-2021 to really be 05-Jan-2021.

What does this have to do with your APEX application? In all my applications I tend to set the default date format to DD-MON-YYYY as it's very explicit. Note: to set the default date format in APEX go to Shared Components > Globalization > Application Date Format. Most of the Date items that I use allow users to either enter the date manually or select from the date picker (via button click). Since entering a two digit year is a valid date neither APEX or Oracle raises an error so it's extremely difficult to catch.

To get around this issue you can change the Application Date Format to: DD-MON-RRRR. From the Oracle documentation:

The RR datetime format element is similar to the YY datetime format element, but it provides additional flexibility for storing date values in other centuries. The RR datetime format element lets you store 20th century dates in the 21st century by specifying only the last two digits of the year.

Using our initial example

select to_date('24-Jan-21', 'DD-MON-YYYY')
from dual;

-- Returns
24-jan-2021

The nice thing about using the RRRR format is you can still display years as four digits but users can enter them in as two digits with expected results. If users are entering past dates (ex: cataloging old library books) I suggest you read the documentation to see if the RRRR format is the right setting for your application.

{% asset_img js-expression.png %}

As the name (JavaScript Expression) suggests this should be an expression such as 1 + 2; If you try to run multiple lines of code with a return statement (as shown above) the following error is raised:

Uncaught TypeError: apex.da.initDaEventList is not a function

To resolve this issue you can use an immediately-invoked Function Expressions (IIFE) as the JavasScript Expression. The following example re-writes the code from the first example as an IIFE and will work as a valid JavaScript Expression in the DA:

(function() {
  var today = new Date();
  return today.getFullYear();
})()

Thanks to Adrian Png and Trent Schafer for the help on this.

This is the desired outcome of the article below. To summarize the solution will allow me to quickly edit a Word template using Office 365. Then view the generated PDF in APEX with a minimal amount of effort.

{% asset_img aop-template.gif %}

Background

AOP has two common uses cases (personal opinion). The first is providing users the ability to download their reports exactly as they're shown in any format they'd like. For example if they build an Interactive Report (IR) with row highlighting etc and click "Download", AOP will make the report look exactly as the user defined it. The second, which this article focuses on, is to create customizable documents based on data in Oracle. A good example of this is generating a invoice for a customer.

Problem

Using the later case, to generate a PDF invoice you need to create a Word or Excel file and use substitution string for AOP to replace data with. For example, to show the Invoice Number based on the column invoice_num you would reference {invoice_num} in your Word document. AOP with then "fill in" the substitution strings with your data. In this case the process is as follows:

1. Define your query (with all the data)
   • One time task unless more data is required
2. Upload Word template file
   • They're various options as to where to store a file (more on this later)
3. Generate the PDF
   • This will involved downloading and then opening the PDF on your laptop

When initially building a template file the last two steps can happen many times. I.e. I want to see what the invoice would look like if I move some text over a bit. I'd need to re-upload the file then re-generate and download the PDF. This process has a few problems:

• It's not good for multiple people working on the same template at the same time
• There's lots of steps involved once making a change to the template to see the resulting PDF

Solution

To resolve the issues described above I created the following process to allow me to quickly build and view invoices using AOP and APEX. These steps require that you have an Office 365 account. (Note: if using a coporate Office 365 you'll need the ability to share files with people outside of your organization. If you're not allowed to use a personal account to host the template file)

Host Template on OneDrive

• Go to One Drive and upload a new Word document
  • Note: For some reason when creating the Word document directly in One Drive AOP can't process the file. To get around this create a Word document on your desktop and upload
  • Note: the same steps can be used for any type of Office document (ex: Excel)
• Generate Share URL
  • This will be a link to give others to work on modifying the Word template file
  • We'll refer to this URL as the "Share URL"
  • Right click on the file and click Share

{% asset_img office-365-share-01.png %}

• A new modal box pops up. Make sure to select the appropriate permissions (not critical that everyone can edit but does make it easy to share the link)
• Select Copy link and save this link somewhere
  • In my example the link is: https://1drv.ms/w/s!AmD6WK_LJIvHq8xyy0JR-JT6ihJdPQ?e=xC88lZ

{% asset_img office-365-share-02.png %}

• Generate Embed URL
  • This will be used by your database to download the template file and send to AOP
  • We'll refer to this URL as the "Embed URL"
  • Right click on the file and click Embed

{% asset_img office-365-embed-01.png %}

• A new slider pops up on the right hand side. Click Generate

{% asset_img office-365-embed-02.png %}

• Copy the iframe HTML tag and save this link
  • In my example the HTML is: <iframe src="https://onedrive.live.com/embed?cid=C78B24CBAF58FA60&resid=C78B24CBAF58FA60%21714354&authkey=AKLKkXB0W5OfUZ4&em=2" width="476" height="288" frameborder="0" scrolling="no"></iframe>

{% asset_img office-365-embed-03.png %}

APEX Setup

Make sure that your APEX application has the latest AOP plugins installed.

Create a new page with a new region. Add the following text page items (Note: you can change to hidden later on / load from a table)

• P3120_ONEDRIVE_EMBED_IFRAME

In the region add the following in the Source > Text box:

<div id="pdf" data-aop-inline-pdf="onClick: Generate PDF" style="width:100%;height:600px;"></div>

{% asset_img apex-setup-region.png %}

Create a new region button called GENERATE_PDF

• Set the Button Position to Edit
• Right click on the button and Create Dynamic Action
  • Name: onClick: Generate PDF
    • Note: Do Not this name must match the data-aop-inline-pdf from the previous div tag
  • Add the following actions:
    • Execute Server Side Code
      • Items to Submit: P3120_ONEDRIVE_EMBED_IFRAME
      • PL/SQL Code:

declare
  l_download_url varchar2(4000);
  l_blob blob;
  l_file_name varchar2(4000);
begin
  -- Need to replace ampersand with amp; for XML parsing
  -- Need to do in SQL as extractvalue not PL/SQL
  select replace(extractvalue(xmltype(replace(:p3120_onedrive_embed_iframe, chr(38), chr(38) || 'amp;')), '/iframe/@src'), 'embed', 'download')
  into l_download_url
  from dual;


  -- Need to download the blob here as AOP expects a filename in the URL. The Office downloads don't have that.

  -- Download the file into a blob
  l_blob := apex_web_service.make_rest_request_b(
    p_url => l_download_url,
    p_http_method => 'GET');

  -- Need to extract filename so that we can get the file extension
  for i in 1.. apex_web_service.g_headers.count loop
    if apex_web_service.g_headers(i).name = 'Content-Disposition' then
      -- Value will look like: attachment; filename="wo-test.docx"
      l_file_name := trim('"' from  regexp_substr(apex_web_service.g_headers(i).value, '".*"'));
      exit; -- Don't need to parse headers anymore
    end if;
  end loop;

  -- Store in Collection for retrieval from AOP
  apex_collection.create_or_truncate_collection(
    p_collection_name => 'AOP_TEMPLATE');

  -- Load
  apex_collection.add_member(
    p_collection_name => 'AOP_TEMPLATE',
    p_c001 => l_file_name,
    p_c002 => apex_string_util.get_file_extension(l_file_name),
    p_blob001 => l_blob
  );
end;

{% asset_img apex-da-execute-plsql.png %}

• Plugin: UC - APEX Office Print (AOP) - DA [Plug-In]
  • Note: You'll to have installed the AOP Plugins for this to show up in list
  • Settings:
    • Template Type: SQL
    • Template Source:

select
  ac.c002 file_ext,
  ac.blob001 blob_contents
from apex_collections ac
where 1=1
  and ac.collection_name = 'AOP_TEMPLATE'

• Data Type: SQL
• Data Source:

-- Change this to reference you tables etc
select
  'demo' as "filename",
  cursor(
    select to_char(sysdate, 'DD-MON-YYYY') as "today"
    from dual
  ) as "data"
from dual

• Output Type: PDF
  • Note you can change this to a different type of file later but for the preview in APEX, PDF is required
• Output To: Inline Region (pdf/html/md/txt only)
  • This is what will make the PDF show up in APEX

{% asset_img apex-da-aop-plugin.png %}

Using the iFrame URL, users should be able to see the preview in APEX based on changes made in Office 365 (see video at top of article)

{% asset_img macos-error.png %}

To resolve this issue I got help from Kris Rice and Niels de Bruijn. The issue is caused by Apple and the Java applet plugin.

Here's how I got SQL Developer working:

List your Java Versions

/usr/libexec/java_home -V

# This will give an output like:

Matching Java Virtual Machines (3):
    11.0.2 (x86_64) "Oracle Corporation" - "Java SE 11.0.2" /Library/Java/JavaVirtualMachines/jdk-11.0.2.jdk/Contents/Home
    1.8.271.09 (x86_64) "Oracle Corporation" - "Java" /Library/Internet Plug-Ins/JavaAppletPlugin.plugin/Contents/Home
    1.8.0_201 (x86_64) "Oracle Corporation" - "Java SE 8" /Library/Java/JavaVirtualMachines/jdk1.8.0_201.jdk/Contents/Home
/Library/Java/JavaVirtualMachines/jdk-11.0.2.jdk/Contents/Home

The one that caused the problem is the "Oracle Corporation" - "Java" /Library/Internet Plug-Ins/JavaAppletPlugin.plugin/Contents/Home and you'll need to remove it.

Remove the Applet Plugin

sudo rm -rf "/Library/Internet Plug-Ins/JavaAppletPlugin.plugin/"


# After deleting this folder it should be removed from the Java Home list
/usr/libexec/java_home -V

# Output
Matching Java Virtual Machines (2):
    11.0.2 (x86_64) "Oracle Corporation" - "Java SE 11.0.2" /Library/Java/JavaVirtualMachines/jdk-11.0.2.jdk/Contents/Home
    1.8.0_201 (x86_64) "Oracle Corporation" - "Java SE 8" /Library/Java/JavaVirtualMachines/jdk1.8.0_201.jdk/Contents/Home
/Library/Java/JavaVirtualMachines/jdk-11.0.2.jdk/Contents/Home

If you run SQL Developer now it should work. I'm not sure about the side effects of removing this Java applet plugin. If I find it to be critical to other things I'll be sure to update this post.

• How to take a screenshot
• Screenshot tool features

One thing that is a bit of a hidden gem is the ability to quickly markup screen shots both using the Markup tool. To activate the markup tool click the thumbnail icon of the screenshot (right after it's taken). The following video highlights this:

{% asset_img macos-screenshot-markup.gif %}

The Markup tool has a very cool feature called Continuity Markup that allows you to markup the screenshot either on your iPad or iPhone. The following video shows this. Note: once I clicked on the thumbnail of my screenshot, in the bottom right corner on my screen, it automatically turned on my iPad for me!

You can also use Continuity Markup when using Quick Look to preview a file (ex: hitting spacebar on a file in Finder).